Part A Convolution Neural Networks

Name	Admin No.	Class
Ernest Ng Sheng Kang	P2222668	DAAA/FT/2B/06

Background Infomation

Convolutional Neural Networks (CNNs) are specialized artificial neural networks designed for processing structured grid data, particularly images. They have become widely popular in computer vision tasks due to their capacity to automatically learn hierarchical features from input data.

Initializing Libaries

import tensorflow as tf
import os
import json
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from PIL import Image
import seaborn as sns
from sklearn.metrics import classification_report
from keras.utils import to_categorical
import keras_tuner as kt
import visualkeras
from keras.regularizers import l1, l2
from keras.layers import AveragePooling2D, ZeroPadding2D, BatchNormalization, Activation, MaxPool2D, Add
from keras.models import Sequential
from keras.layers import Dropout
from keras.layers import Flatten
from keras.layers import Conv2D
from keras.layers import MaxPooling2D
from keras.layers import Normalization, Dense, Conv2D, Dropout, BatchNormalization, ReLU
from keras.models import Sequential
from keras.models import Model
from tensorflow.keras.preprocessing.image import ImageDataGenerator
from keras import Input
from keras.optimizers import *
from keras.callbacks import EarlyStopping

Using TensorFlow backend

Checking GPU

tf.config.list_physical_devices('GPU')

[PhysicalDevice(name='/physical_device:GPU:0', device_type='GPU')]

Fixing random seed

Setting the seed for reproducibilty

seed = 69
np.random.seed(seed)
tf.random.set_seed(seed)

Problem statement

Our problem for this assignment is to create a deep learning model to predict vegatables using the dataset on vegatables.

We need to create 2 seperate final models to predict images on 2 different input sizes:

• 31 x 31
• 128 x128

Goals of this project

Generalise to data well

To make sure our model is able to generalise to data well, we need to minimize the difference between our training accuracy and test accuracy.

Accuracy

In order to minimize avoidable bias, it is important for the model to achieve a training set accuracy of at least 85% or higher, which signifies a reasonable level of accuracy.

Importing Datasets

directory_train = './datasets/Dataset for CA1 part A/train'
directory_test = './datasets/Dataset for CA1 part A/test'
directory_validation = './datasets/Dataset for CA1 part A/validation'

def dataset(directory,image_size,seed=seed,batch_size=64,shuffle=True):
    dataset = tf.keras.preprocessing.image_dataset_from_directory(
        directory,
        labels='inferred',
        label_mode='categorical',
        color_mode='grayscale',
        image_size=image_size,
        shuffle=shuffle,
        seed=seed,
        batch_size=batch_size
    )
    return dataset

dataset_train_31 = dataset(directory_train,(31,31))
dataset_test_31 = dataset(directory_test,(31,31))
dataset_validation_31 = dataset(directory_validation,(31,31))

dataset_train_128 = dataset(directory_train,(128,128))
dataset_test_128 = dataset(directory_test,(128,128))
dataset_validation_128 = dataset(directory_validation,(128,128))

Found 9028 files belonging to 15 classes.
Found 3000 files belonging to 15 classes.
Found 3000 files belonging to 15 classes.
Found 9028 files belonging to 15 classes.
Found 3000 files belonging to 15 classes.
Found 3000 files belonging to 15 classes.

EDA

What can we get from our EDA?

• How does the average image look like for each class?
• How is the class distribution?
• How does a sample image look like for each class?

class_labels = [
    "Bean",
    "Bitter Gourd",
    "Bottle Gourd",
    "Brinjal",
    "Broccoli",
    "Cabbage",
    "Capsicum",
    "Carrot",
    "Cauliflower",
    "Cucumber",
    "Papaya",
    "Potato",
    "Pumpkin",
    "Radish",
    "Tomato",
]

for images, labels in dataset_train_31.take(1):
    print("31x31 Batch Shape:", images.shape)
for images, labels in dataset_train_128.take(1):
    print("128x128 Batch Shape:", images.shape)

31x31 Batch Shape: (64, 31, 31, 1)
128x128 Batch Shape: (64, 128, 128, 1)

Our image shapes are 31x31 and 128x128, with a single colour channel.

We have a default batch size of 64, since tf.keras.preprocessing.image_dataset_from_directory requires a default batch size or else the model wont fit properly. Fortunatly we can set our batch size again when we fit our models.

Visualizing our datasets

def plot_unique_images(dataset, title, class_labels):
    plt.figure(figsize=(12, 6))
    plt.suptitle(title, fontsize=20)
    unique_images = []  
    unique_labels = []  

    #get unique images
    for images, labels in dataset:
        for i, label in enumerate(labels):
            label_idx = tf.argmax(label).numpy()
            if label_idx not in unique_labels:
                unique_images.append(images[i].numpy().astype("uint8"))
                unique_labels.append(label_idx)

            if len(unique_images) == len(class_labels):
                break 
        if len(unique_images) == len(class_labels):
            break  

    num_images = len(unique_images)
    row = int(np.sqrt(num_images))
    col = int(np.ceil(num_images / row))

    for i in range(num_images):
        ax = plt.subplot(row, col, i + 1)
        plt.imshow(unique_images[i], cmap='gray')
        plt.title(class_labels[unique_labels[i]])
        plt.axis("off")

    plt.tight_layout()
    plt.show()

plot_unique_images(dataset_train_128, '128x128 Images for Each Class', class_labels)

plot_unique_images(dataset_train_31, '31x31 Images for Each Class', class_labels)

Observations:

• We can hardly differentiate some images between classes for images of size 31x31(brocoli,bitter_gourd), hopefully our model can better differeniate these images better than humans!

Now lets get a wider sample range of pictures of every class, to see how diverse the classes are.

def plot_dataset_huge(amtOfImages, dataset,title,class_labels):
    plt.figure(figsize=(12, 10))
    plt.suptitle(title, fontsize=20)
    for images, labels in dataset.take(1):
        num_images = len(images)  

        row = int(np.sqrt(amtOfImages))
        col = int(np.ceil(amtOfImages / row))
            
        for i in range(min(amtOfImages, num_images)):
            ax = plt.subplot(row, col, i + 1)
            plt.imshow(images[i].numpy().astype("uint8"), cmap='gray')
            plt.title(class_labels[np.argmax(labels[i])])
            plt.axis("off")

    plt.subplots_adjust(wspace=0.1, hspace=0.01)
    plt.tight_layout()
    plt.show()

plot_dataset_huge(48,dataset_train_128,'128 x 128 images',class_labels)

plot_dataset_huge(48,dataset_train_31,'31 x 31 images',class_labels)

Observations:

• Cauliflowers and Potatos are easily differentiable, with potatos usually having a white background and cauliflowers having

Class Distribution

class_counts_df = pd.DataFrame(columns=class_labels)

def count_classes(dataset, class_labels):
    class_count = {label: 0 for label in class_labels}
    for _, labels in dataset:
        for label in labels:
            class_name = class_labels[tf.argmax(label)]
            class_count[class_name] += 1
    return class_count

class_counts_df.loc['Validation'] = count_classes(dataset_validation_128, class_labels)
class_counts_df.loc['Training'] = count_classes(dataset_train_128, class_labels)
class_counts_df.loc['Test'] = count_classes(dataset_test_128, class_labels)
class_counts_df.loc['Total Classes'] = class_counts_df.sum()
class_counts_df['Total Datasets'] = class_counts_df.sum(axis=1)

class_counts_df

	Bean	Bitter Gourd	Bottle Gourd	Brinjal	Broccoli	Cabbage	Capsicum	Carrot	Cauliflower	Cucumber	Papaya	Potato	Pumpkin	Radish	Tomato	Total Datasets
Validation	200	200	200	200	200	200	200	200	200	200	200	200	200	200	200	3000
Training	780	720	441	868	750	503	351	256	587	812	566	377	814	248	955	9028
Test	200	200	200	200	200	200	200	200	200	200	200	200	200	200	200	3000
Total Classes	1180	1120	841	1268	1150	903	751	656	987	1212	966	777	1214	648	1355	15028

class_counts_df_plot = class_counts_df.drop(['Total Datasets'], axis=1)
class_counts_df_plot = class_counts_df_plot.T
class_counts_df_plot

# Plot the bar graph
class_counts_df_plot['Total Classes'].plot(kind='bar', figsize=(17, 6))
plt.xlabel('Classes')
plt.ylabel('Total Images')
plt.title('Class Distribution')
plt.xticks(rotation=45)  # Rotate x-axis labels for better readability
plt.show()

We can see that we have an class imbalance for our train dataset. Oh No!! What should we do?

• There are quite alot of solutions for this problem, here are 3 main solutions:
  1. We can augment our data on the minority classes(Bottle_gourd, carrots and radish). However we want to also augment our other images classes to prevent overfitting so this is not a good option.
  2. We can oversample or undersample our minority or majority classes.However oversampling can lead to overfitting without any data augmentation.
  3. We can modify the loss function to assign different weights to each class, such as increasing the loss weigh for our minority class.

Why will we be using differing class weights?

We will be assigning different weights for each class for a few reasons:

• It helps preserve our existing data, as in our case we have classes that have a much smaller number of samples (Carrots and Radishs) compared to others.We effectively tell the model to pay more attention to the minority classes.
• Prevents overfitting. As our model is encouraged to strike a balance between minimizing the loss and ensuring that all our classes are predicted accurately.
• Simpler than Image oversampling. Using differing class weights can be more advantagous than image oversampling due to its increased computational costs and potential for overfitting without image augmentation.

class_counts_df_plot['Training'].values

array([780, 720, 441, 868, 750, 503, 351, 256, 587, 812, 566, 377, 814,
       248, 955], dtype=int64)

We will be calculating the class weights using only the train data, as it is the only dataset with imbalanced classes.

class_weights = tf.math.reciprocal(tf.constant(
(class_counts_df_plot['Training'].values).astype(np.float32)))

# Normalize the class weights
class_weights = class_weights / tf.reduce_sum(class_weights)
class_weights_dict = {i: weight for i, weight in enumerate(class_weights.numpy())}

print("Class Weights:", class_weights_dict)

Class Weights: {0: 0.042979263, 1: 0.046560865, 2: 0.07601774, 3: 0.038621914, 4: 0.04469843, 5: 0.06664775, 6: 0.09550946, 7: 0.13095243, 8: 0.05711043, 9: 0.041285496, 10: 0.05922937, 11: 0.088922605, 12: 0.041184057, 13: 0.1351767, 14: 0.03510348}

Another important question to ask is.

• What are our pixel distributions for our datasets? and what is the std and mean of them?

def compute_statistics(dataset,title):
    pixel_min = float('inf') 
    pixel_max = float('-inf')
    pixel_sum = 0
    pixel_sum_square = 0
    num_pixels = 0

    for images, _ in dataset:
        for image in images:
            # Calculate min and max pixel values
            pixel_min = min(pixel_min, np.min(image))
            pixel_max = max(pixel_max, np.max(image))

            # Update sum and sum of squares for mean and std calculation
            pixel_sum += np.sum(image)
            pixel_sum_square += np.sum(image ** 2)
            num_pixels += np.prod(image.shape)

    # Calculate mean and std
    pixel_mean = pixel_sum / num_pixels
    pixel_std = np.sqrt((pixel_sum_square / num_pixels) - (pixel_mean ** 2))

    print(title)
    print("Min Pixel Value:", pixel_min)
    print("Max Pixel Value:", pixel_max)
    print("Mean Pixel Value:", pixel_mean)
    print("Standard Deviation:", pixel_std)

    return

compute_statistics(dataset_train_31,'Training Images')
compute_statistics(dataset_test_31,'Test Images')
compute_statistics(dataset_validation_31,'Validation Images')

Training Images
Min Pixel Value: 0.0
Max Pixel Value: 255.0
Mean Pixel Value: 114.34127948859215
Standard Deviation: 55.387912533676406
Test Images
Min Pixel Value: 0.0
Max Pixel Value: 255.0
Mean Pixel Value: 114.66371605098855
Standard Deviation: 55.07995207606749
Validation Images
Min Pixel Value: 0.0
Max Pixel Value: 255.0
Mean Pixel Value: 113.78344607803287
Standard Deviation: 55.49623592897191

• Our pixel values range from 0 to 255, which is expected as grayscale images typically have pixel values ranging from 0 (black) to 255 (white).
• Our average pixel is around 114, which means that our images are not heavily skewed toward either very dark or very bright values.

Image Averaging

We will be using only the test and validation datasets for averaging as the train set has class imbalance and may result in inaccurate results.

def calculate_mean_image(datasets, image_size, title):
    pixel_sum = np.zeros((image_size[0], image_size[1]), dtype=np.float64)
    num_images = 0
    mean_images = np.zeros((image_size[0], image_size[1]), dtype=np.float64)

    for dataset in datasets:
        for images, _ in dataset:
            for image in images:
                pixel_sum += np.squeeze(image, axis=-1)
                num_images += 1

    # Calculate the mean image
    mean_image = pixel_sum / num_images
    mean_images = mean_images + mean_image
    
    mean_images = mean_images / len(datasets)

    return mean_images

image_size_31 = (31, 31)
image_size_128 = (128, 128)

mean_image_31 = calculate_mean_image([dataset_test_31, dataset_validation_31], image_size_31, 'Average Mean Image (31x31)')
mean_image_128 = calculate_mean_image([dataset_test_128, dataset_validation_128], image_size_128, 'Average Mean Image (128x128)')

plt.figure(figsize=(10, 5))
plt.subplot(1, 2, 1)
plt.imshow(mean_image_31, cmap='gray')
plt.title('31x31 Mean Image')
plt.axis("off")
plt.subplot(1, 2, 2)
plt.imshow(mean_image_128, cmap='gray')
plt.title('128x128 Mean Image')
plt.axis("off")
plt.tight_layout()
plt.show()

The mean image for all the images combined looks like a fog, and we cant really determine any of or classes from this average image.

def calculate_class_mean_images(dataset, class_labels, image_size, title,amtOfImages=15):
    class_mean_images = {}  # To store the mean images for each class

    # Initialize mean images for each class
    for class_label in class_labels:
        class_mean_images[class_label] = np.zeros((image_size[0], image_size[1]), dtype=np.float64)

    num_images = 0

    for images, labels in dataset:
        for i, label in enumerate(labels):
            class_label = class_labels[np.argmax(label)]
            class_mean_images[class_label] += np.squeeze(images[i], axis=-1)
            num_images += 1

    num_images = amtOfImages
    row = int(np.sqrt(num_images))
    col = int(np.ceil(num_images / row))

    # Calculate and display the mean image for each class
    plt.figure(figsize=(10, 6))
    plt.suptitle(title, fontsize=20)

    for i, class_label in enumerate(class_labels):
        mean_image = class_mean_images[class_label] / (class_mean_images[class_label].max() + 1e-8)
        plt.subplot(row,col, i + 1)
        plt.imshow(mean_image, cmap='gray')
        plt.title(class_label)
        plt.axis("off")

    plt.tight_layout()
    plt.show()

calculate_class_mean_images(dataset_test_128, class_labels, image_size_128, 'Average Mean Images (128x128)')
calculate_class_mean_images(dataset_test_31, class_labels, image_size_31, 'Average Mean Images (31x31)')

Observations:

• We can see that the pixels for different classes kind of group together to show where the vegetable is. For example, the average image for potato has a cluster of black pixels in the center, roughly outlining a potato.
• However as the images in the dataset are quite complicated, it is quite hard to differentiate out individual classes without labels.

Data Preprocessing

We need to preprocess our data to help our models to converge faster ,ie lesser time to train. And helps the model to generalise better and prevent it from overfitting.

Normalizing our images

Why do we need to normalize our images?

As normalizing involves scalling the pixel values of our images between 0 and 1 , this results in our images all having a similiar data distribution. Which makes our models benefit from faster convergence during gradient descent and generalising better.

Normalizing the inputs in our context involves calculating the mean and standard deviation of the training dataset and then applying the following formula to each input: $$ X_{} = \frac{X_{} - μ_{}}{σ_{}} $$

# use rescaling
normalization_layer = tf.keras.layers.Rescaling(1./255)
normalized_dataset_train_31 = dataset_train_31.map(lambda x, y: (normalization_layer(x), y))

compute_statistics(normalized_dataset_train_31,'Normalised Training Images')

Normalised Training Images
Min Pixel Value: 0.0
Max Pixel Value: 1.0
Mean Pixel Value: 0.448397200505043
Standard Deviation: 0.21720750670021785

We can see that the range of the new normalised pixel values is between 0 - 1

for images, labels in dataset_train_31.take(1):
    original_images = images

normalized_images = normalization_layer(original_images)

original_image = original_images[0]
normalized_image = normalized_images[0]

fig, ax = plt.subplots(ncols=2, figsize=(10, 5))

ax[0].imshow(original_image.numpy().squeeze(), cmap='gray')
ax[0].set_title('Before Normalization')

ax[1].imshow(normalized_image.numpy().squeeze(), cmap='gray')
ax[1].set_title('After Normalization')

plt.show()

Observations:

• We can see that normalization dosent significantly affect the visual appearance of our image. Since the original pixel values were already within the same range as before.

Data Augementation

What are image augmentations?

It is a technique that artificially enhances the diversty and variability of our image dataset. It involves applying various transformations to our images such as rotations, flips, scaling, brightness adjustments, and cropping. It is mainly to provide our model with a broader range of training examples which improves its abilty to generalise and predict better on unseen data.

Why do we need to augment our images?

There are serveral reasons why we need to augment our images.

• 1. It increases the amount of images in our train dataset, thus having more "data" for the model to train on.
• 2. It helps with reducing overfitting,by exposing our model to a wider range of inputs, which promotes robustness and enhances its ability to handle variations.

Types of Image Augmentations we will be using:

• flipping (horizontal/ vertically)
• cropping

def augment_images(image, label, size, seed, flip_lr_prob=0.2, flip_ud_prob=0.2):
    image = tf.map_fn(lambda img: tf.image.random_crop(img, size, seed=seed), image)

    # Randomly flip the image left or right based on the flip_lr_prob
    image = tf.cond(tf.random.uniform([]) < flip_lr_prob, 
                    lambda: tf.image.random_flip_left_right(image, seed=seed), 
                    lambda: image)
    # Randomly flip the image up or down based on the flip_ud_prob
    image = tf.cond(tf.random.uniform([]) < flip_ud_prob, 
                    lambda: tf.image.random_flip_up_down(image, seed=seed), 
                    lambda: image)

    return image, label

size_31 = [31, 31, 1] 
size_128 = [128, 128, 1]
train_generator_31 = dataset_train_31.map(lambda image, label: augment_images(image, label, size_31,seed=seed))
train_generator_128 = dataset_train_128.map(lambda image, label: augment_images(image, label, size_128,seed=seed))
#plot the augmented images

We won't have servere data augmentation, as from my testing, servere data augmentation tends to make the model training very unstable and it sometimes even makes the model worse when generalising to our validation data.

plot_dataset_huge(42,train_generator_128,'Augmented 128 x 128 images',class_labels)

plot_dataset_huge(42,train_generator_31,'Augmented 31 x 31 images',class_labels)

Observations:

• We have successfully augmented some of our images, it is visible that some of the images has been flipped and cropped. This will hopefully introduce more variants into our dataset and enable our model to better generalise to our test and validation datasets.

Models

How does a CNN differ from a ANN?

As the name of a CNN suggests, its convolution! As image data is store in arrays, It is deeply inefffcient and unsatisfying to flatten the whole image into an input layer and just feed it through a neural network. With neural network that recognises images having millions of tunable parameters. CNNs help solve that issue, with convolution and pooling layers!

How does each layer affect our accuracy and efficiency?

We have serveral layers we want to look into here:

Convolution Layer(Convo2D)

• They are basically require in our CNN model to extract the features in our images and can learn hierarchical features.
• However having too much convolution layers without regularization can lead to overfitting :(
• Convolutional layers are computationally intensive, but their use of shared weights reduces the overall number of parameters compared to fully connected layers, makes them computationally more efficient. :)

Pooling layer

• Pooling layers downsample feature maps from Convolution layers, reducing its spatial dimensions and computational requirements. :)
• However excessive pooling may lead to information loss, affecting the model's ability to localize features precisely. :(
• It increases computational efficiency by reducing the spatial dimensions, which can lead to faster training times and convergence. :)

Fully connected layer (Dense)

• They are the main "brain" of our CNN, they are responsible for combining extracted features for final classification and can capture complex relationships in the data. :)
• Too many parameters in dense layers can lead to overfitting, especially if our dataset is small which in our case it is. :(
• Dense layers are alot more computationally expensive due to the large number of parameters, possibly slowing down training times. :(

Dropout layer

• Dropout layers prevent overfitting by randomly deactivating a fraction of neurons during training, encouraging our CNN to learn more robust features. :)
• However if the dropout layer is too aggressive, it may prevent the model to learn any meaningful features or patterns, lowering the accuracy. :(

Models choosen

We will be experimenting on different CNN architechtures on our 31 x 31 and 128 x 128 images

• Convolution 2d Neural Network (base)
• Custom VG16 Network
• Custom ResNet Network

We will be comparing our Custom VG16 Network and Custom Resnet Network with our base model and see how it performs against it.

Choosing Optimization Algorithm

Our choice of optimization algorithms plays a huge role in training CNN models. The 2 most common optimizers used in TensorFlow are Adam and Stochastic Gradient Descent (SGD). We first need to understand how these 2 optimization algorithms function:

Adam Optimzer

• Adam optimization is a stochastic gradient descent method that adapts its learning rates for each network weight individually. It is based on adaptive estimation of first-order (mean) and second-order (uncentered variance) moments. Adam is known for its fast convergence and efficiency in handling noisy or sparse gradients [1].

SGD

• SGD is an method that updates model parameters based on the gradient of the loss function with respect to those parameters. It is a classic optimization algorithm and is known for its simplicity [2].

Our choosen Optimization Algorithm : SGD

Why SGD? Eventhough Adam is faster in many cases as compared to SGD, this is due to it updating its learning rates indivdually for each weight and techniques such as coordinate-wise gradient clipping to handle noise effectivly. On the other hand, SGD may out perform Adam in image classification tasks, as Adam takes "shortcuts" where it dosent preserve the fine-grained details to distingush an image.

Therefore we will be using SGD as our optimizer

Overfitting prevention

To prevent overfitting, we will be using Early Stopping, as it will stop model training once it begins to overfit. Other strategies include:

• Data Augmentation
• Normalisation
• Dropout layers
• L1 and L2 Regularization

Does size matter?

Does our input image size really matter? what else were you thinking!?

• Larger input sizes requires more complex models to capture intricate patterns and details in the data. We may need deeper convolution networks to better learn features from our 128 x 128 pictures
• Smaller input sizes eg 31x31 might result in models that generalize better. Larger inputs eg 128x128 can lead to overfitting, where the model may memorize details specific to the training set but fail to generalize to new data.

In summary, smaller sizes tend to give faster training times as models with fewer parameters converge more swiftly during optimization and is able to generalise better. On the other hand, larger sizes introduce increased model complexity, which enables the capture of the finner details and possibly improving performance on tasks requiring fine-grained information. However, this advantage comes at the cost of longer training times and heightened computational demands.

Utilty Functions

def compare_curves(history1, history2, title1, title2):
    plt.figure(figsize=(16, 6))

    # Loss
    plt.subplot(1, 2, 1)
    plt.plot(history1['loss'], 'o-', label='Train Loss ' + title1)
    plt.plot(history1['val_loss'], 'o-', label='Validation Loss ' + title1)
    plt.plot(history2['loss'], '*--', label='Train Loss ' + title2)
    plt.plot(history2['val_loss'], '*--', label='Validation Loss ' + title2)
    plt.title('Loss Curve Comparison')
    plt.xlabel('Epochs')
    plt.ylabel('Loss')
    plt.ylim([0, 3])
    plt.legend()

    # Accuracy
    plt.subplot(1, 2, 2)
    plt.plot(history1['accuracy'], 'o-', label='Train Accuracy ' + title1)
    plt.plot(history1['val_accuracy'], 'o-', label='Validation Accuracy ' + title1)
    plt.plot(history2['accuracy'], '*--', label='Train Accuracy ' + title2)
    plt.plot(history2['val_accuracy'], '*--', label='Validation Accuracy ' + title2)
    plt.title('Accuracy Curve Comparison')
    plt.xlabel('Epochs')
    plt.ylabel('Accuracy')
    plt.ylim([0, 1])
    plt.legend()

    plt.tight_layout()
    plt.show()

def plot_history(history, title):
    plt.figure(figsize=(16, 6))

    # Loss
    plt.subplot(1, 2, 1)
    plt.plot(history['loss'], 'o-', label='Train Loss')
    plt.plot(history['val_loss'], 'o-', label='Validation Loss')
    plt.title('Loss Curve - ' + title)
    plt.xlabel('Epochs')
    plt.ylabel('Loss')
    plt.ylim([0, 3])
    plt.legend()

    # Accuracy
    plt.subplot(1, 2, 2)
    plt.plot(history['accuracy'], 'o-', label='Train Accuracy')
    plt.plot(history['val_accuracy'], 'o-', label='Validation Accuracy')
    plt.title('Accuracy Curve - ' + title)
    plt.xlabel('Epochs')
    plt.ylabel('Accuracy')
    plt.ylim([0, 1])
    plt.legend()

    plt.tight_layout()
    plt.show()

def evaluate_and_save_model(model, model_name, batch_size, save_dir='./models',save_models=False):
    # Create directory if it doesn't exist
    if save_models:
        if not os.path.exists(save_dir):
            os.makedirs(save_dir)
        # Save the model
        model_path = os.path.join(save_dir, model_name + '.h5')
        #replace the model if it already exists
        if os.path.exists(model_path):
            os.remove(model_path)
        model.save(model_path)
        print('Saved trained model at %s ' % model_path)

        model_history = model.history.history
        if os.path.exists(os.path.join(save_dir, model_name +'_history'+'.json')):
            os.remove(os.path.join(save_dir, model_name + '_history'+'.json'))
        json.dump(model_history, open(os.path.join(save_dir, model_name +'_history'+ '.json'), 'w'))
        print('Saved trained model history at %s ' % os.path.join(save_dir, model_name +'_history'+ '.json'))
    else:
        model_history = model.history.history

    best_val_idx = np.argmax(model_history["val_accuracy"])
    result = {}
    result["Model Name"] = model_name
    result["Epochs"] = len(model_history["loss"])
    result["Batch Size"] = batch_size
    result["Train Loss"] = model_history["loss"][best_val_idx]
    result["Val Loss"] = model_history["val_loss"][best_val_idx]
    result["Train Acc"] = model_history["accuracy"][best_val_idx]
    result["Val Acc"] = model_history["val_accuracy"][best_val_idx]
    result["[Train - Val] Acc"] = result["Train Acc"] - result["Val Acc"]
    result["[Train - Val] Loss"] = result["Train Loss"] - result["Val Loss"]
    result_series = pd.Series(result, name=result["Model Name"])
    print(result_series)
    return

from keras import backend as K
"""
source : https://stackoverflow.com/questions/47680984/does-input-dimension-resolution-affect-the-performance-of-convolutional-neural-n
"""
def recall_m(y_true, y_pred):
    true_positives = K.sum(K.round(K.clip(y_true * y_pred, 0, 1)))
    possible_positives = K.sum(K.round(K.clip(y_true, 0, 1)))
    recall = true_positives / (possible_positives + K.epsilon())
    return recall

def precision_m(y_true, y_pred):
    true_positives = K.sum(K.round(K.clip(y_true * y_pred, 0, 1)))
    predicted_positives = K.sum(K.round(K.clip(y_pred, 0, 1)))
    precision = true_positives / (predicted_positives + K.epsilon())
    return precision

def f1_m(y_true, y_pred):
    precision = precision_m(y_true, y_pred)
    recall = recall_m(y_true, y_pred)
    return 2*((precision*recall)/(precision+recall+K.epsilon()))

Setting Parameters for our model

We will be setting the basic parameters for training our model.

• weight decay
• steps per epoch
• learning rate
• momentum

weight_decay = 0.0005
steps_per_epoch = len(dataset_train_31) # 9028(total training samples) // 64(batch size) = 142
learning_rate = 0.02
momentum = 0.9

Convo2D Neural Network (Base)

We will be using this model as a base to compare our other 2 models in later down the project. It will be a simple model with 2Convo2D and MaxPooling pairs and 2 hidden dense layers each with 128 neurons. The ReLu activation function will be applyed for the hidden layers and softmax for the output layer. As there are multiple category that we are predicting, we will be using the categorical_crossentropy as our loss function. We will be using both F1 and accuracry to measure our model.

Training without Data Augmentation for both 31x31 and 128x128

def create_Basemodel(input_shape, learning_rate, momentum, class_weights_dict, dataset_train, dataset_test, steps_per_epoch):
    tf.keras.backend.clear_session()
    model = Sequential()
    model.add(tf.keras.layers.Rescaling(1./255, input_shape=input_shape))
    model.add(Conv2D(64,(3,3),activation='relu'))
    model.add(MaxPooling2D((2,2)))
    model.add(Conv2D(64,(3,3),activation='relu'))
    model.add(MaxPooling2D((2,2)))
    model.add(Flatten())
    model.add(Dense(128, activation='relu'))
    model.add(Dense(128, activation='relu'))
    model.add(Dense(15, activation='softmax'))
    model.compile(optimizer=SGD(learning_rate=learning_rate, momentum=momentum), loss='categorical_crossentropy', metrics=['accuracy',f1_m])
    model.fit(dataset_train, epochs=150, validation_data=dataset_test, class_weight=class_weights_dict,callbacks=[EarlyStopping(monitor='val_loss', patience=10, verbose=1, mode='min')],steps_per_epoch=steps_per_epoch)
    return model

%%time
BaseModel_31 = create_Basemodel((31,31,1),learning_rate,momentum,class_weights_dict,dataset_train_31,dataset_test_31,steps_per_epoch)

Epoch 1/150
142/142 [==============================] - 3s 12ms/step - loss: 0.1508 - accuracy: 0.0857 - f1_m: 0.0000e+00 - val_loss: 2.7046 - val_accuracy: 0.0933 - val_f1_m: 0.0000e+00
Epoch 2/150
142/142 [==============================] - 2s 10ms/step - loss: 0.1506 - accuracy: 0.1272 - f1_m: 0.0000e+00 - val_loss: 2.7007 - val_accuracy: 0.0870 - val_f1_m: 0.0000e+00
Epoch 3/150
142/142 [==============================] - 2s 10ms/step - loss: 0.1504 - accuracy: 0.1293 - f1_m: 0.0000e+00 - val_loss: 2.6964 - val_accuracy: 0.1013 - val_f1_m: 0.0000e+00
Epoch 4/150
142/142 [==============================] - 2s 11ms/step - loss: 0.1501 - accuracy: 0.1331 - f1_m: 0.0000e+00 - val_loss: 2.6904 - val_accuracy: 0.1183 - val_f1_m: 0.0000e+00
Epoch 5/150
142/142 [==============================] - 2s 12ms/step - loss: 0.1497 - accuracy: 0.1504 - f1_m: 0.0000e+00 - val_loss: 2.6813 - val_accuracy: 0.1360 - val_f1_m: 0.0000e+00
Epoch 6/150
142/142 [==============================] - 2s 12ms/step - loss: 0.1491 - accuracy: 0.1639 - f1_m: 0.0000e+00 - val_loss: 2.6666 - val_accuracy: 0.1553 - val_f1_m: 0.0000e+00
Epoch 7/150
142/142 [==============================] - 2s 11ms/step - loss: 0.1480 - accuracy: 0.1742 - f1_m: 0.0000e+00 - val_loss: 2.6398 - val_accuracy: 0.1577 - val_f1_m: 0.0000e+00
Epoch 8/150
142/142 [==============================] - 2s 11ms/step - loss: 0.1461 - accuracy: 0.1868 - f1_m: 0.0000e+00 - val_loss: 2.5930 - val_accuracy: 0.1863 - val_f1_m: 0.0000e+00
Epoch 9/150
142/142 [==============================] - 2s 11ms/step - loss: 0.1429 - accuracy: 0.1938 - f1_m: 0.0000e+00 - val_loss: 2.5172 - val_accuracy: 0.1957 - val_f1_m: 0.0000e+00
Epoch 10/150
142/142 [==============================] - 2s 11ms/step - loss: 0.1383 - accuracy: 0.2168 - f1_m: 0.0015 - val_loss: 2.4234 - val_accuracy: 0.2323 - val_f1_m: 0.0033
Epoch 11/150
142/142 [==============================] - 2s 11ms/step - loss: 0.1339 - accuracy: 0.2376 - f1_m: 0.0028 - val_loss: 2.3446 - val_accuracy: 0.2570 - val_f1_m: 0.0163
Epoch 12/150
142/142 [==============================] - 2s 11ms/step - loss: 0.1300 - accuracy: 0.2623 - f1_m: 0.0052 - val_loss: 2.2912 - val_accuracy: 0.2693 - val_f1_m: 0.0130
Epoch 13/150
142/142 [==============================] - 2s 11ms/step - loss: 0.1267 - accuracy: 0.2745 - f1_m: 0.0060 - val_loss: 2.2317 - val_accuracy: 0.2873 - val_f1_m: 0.0180
Epoch 14/150
142/142 [==============================] - 2s 11ms/step - loss: 0.1237 - accuracy: 0.2933 - f1_m: 0.0120 - val_loss: 2.1760 - val_accuracy: 0.3007 - val_f1_m: 0.0194
Epoch 15/150
142/142 [==============================] - 2s 12ms/step - loss: 0.1203 - accuracy: 0.3137 - f1_m: 0.0163 - val_loss: 2.1599 - val_accuracy: 0.3137 - val_f1_m: 0.0587
Epoch 16/150
142/142 [==============================] - 2s 11ms/step - loss: 0.1158 - accuracy: 0.3379 - f1_m: 0.0324 - val_loss: 2.0303 - val_accuracy: 0.3553 - val_f1_m: 0.0777
Epoch 17/150
142/142 [==============================] - 2s 11ms/step - loss: 0.1120 - accuracy: 0.3581 - f1_m: 0.0546 - val_loss: 2.0956 - val_accuracy: 0.3327 - val_f1_m: 0.1427
Epoch 18/150
142/142 [==============================] - 2s 10ms/step - loss: 0.1077 - accuracy: 0.3761 - f1_m: 0.0916 - val_loss: 1.8824 - val_accuracy: 0.4090 - val_f1_m: 0.1070
Epoch 19/150
142/142 [==============================] - 2s 10ms/step - loss: 0.1035 - accuracy: 0.4024 - f1_m: 0.1321 - val_loss: 1.8991 - val_accuracy: 0.3843 - val_f1_m: 0.1372
Epoch 20/150
142/142 [==============================] - 2s 11ms/step - loss: 0.1023 - accuracy: 0.4081 - f1_m: 0.1602 - val_loss: 1.7731 - val_accuracy: 0.4383 - val_f1_m: 0.1983
Epoch 21/150
142/142 [==============================] - 2s 11ms/step - loss: 0.0973 - accuracy: 0.4317 - f1_m: 0.1945 - val_loss: 1.7232 - val_accuracy: 0.4397 - val_f1_m: 0.2384
Epoch 22/150
142/142 [==============================] - 2s 11ms/step - loss: 0.0931 - accuracy: 0.4580 - f1_m: 0.2443 - val_loss: 1.6722 - val_accuracy: 0.4627 - val_f1_m: 0.2828
Epoch 23/150
142/142 [==============================] - 2s 11ms/step - loss: 0.0907 - accuracy: 0.4652 - f1_m: 0.2790 - val_loss: 1.5934 - val_accuracy: 0.4860 - val_f1_m: 0.3227
Epoch 24/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0868 - accuracy: 0.4900 - f1_m: 0.3245 - val_loss: 1.5445 - val_accuracy: 0.5060 - val_f1_m: 0.3501
Epoch 25/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0835 - accuracy: 0.5137 - f1_m: 0.3670 - val_loss: 1.6193 - val_accuracy: 0.4717 - val_f1_m: 0.3807
Epoch 26/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0817 - accuracy: 0.5177 - f1_m: 0.3984 - val_loss: 1.6497 - val_accuracy: 0.4720 - val_f1_m: 0.3620
Epoch 27/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0809 - accuracy: 0.5359 - f1_m: 0.4053 - val_loss: 1.4523 - val_accuracy: 0.5217 - val_f1_m: 0.4362
Epoch 28/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0739 - accuracy: 0.5692 - f1_m: 0.4750 - val_loss: 1.3419 - val_accuracy: 0.5790 - val_f1_m: 0.4878
Epoch 29/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0708 - accuracy: 0.5879 - f1_m: 0.4990 - val_loss: 1.3152 - val_accuracy: 0.5870 - val_f1_m: 0.5143
Epoch 30/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0711 - accuracy: 0.5833 - f1_m: 0.5104 - val_loss: 1.2580 - val_accuracy: 0.6033 - val_f1_m: 0.5406
Epoch 31/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0649 - accuracy: 0.6214 - f1_m: 0.5589 - val_loss: 1.4585 - val_accuracy: 0.5423 - val_f1_m: 0.4957
Epoch 32/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0638 - accuracy: 0.6256 - f1_m: 0.5720 - val_loss: 1.1944 - val_accuracy: 0.6083 - val_f1_m: 0.5562
Epoch 33/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0581 - accuracy: 0.6558 - f1_m: 0.6111 - val_loss: 1.2875 - val_accuracy: 0.5810 - val_f1_m: 0.5446
Epoch 34/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0568 - accuracy: 0.6640 - f1_m: 0.6281 - val_loss: 1.2034 - val_accuracy: 0.6213 - val_f1_m: 0.5944
Epoch 35/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0550 - accuracy: 0.6763 - f1_m: 0.6432 - val_loss: 1.1077 - val_accuracy: 0.6557 - val_f1_m: 0.6175
Epoch 36/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0521 - accuracy: 0.6905 - f1_m: 0.6645 - val_loss: 1.0116 - val_accuracy: 0.6860 - val_f1_m: 0.6621
Epoch 37/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0492 - accuracy: 0.7105 - f1_m: 0.6805 - val_loss: 1.2376 - val_accuracy: 0.5913 - val_f1_m: 0.5691
Epoch 38/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0503 - accuracy: 0.7033 - f1_m: 0.6727 - val_loss: 0.9951 - val_accuracy: 0.6860 - val_f1_m: 0.6704
Epoch 39/150
142/142 [==============================] - 2s 10ms/step - loss: 0.0455 - accuracy: 0.7251 - f1_m: 0.7088 - val_loss: 0.9831 - val_accuracy: 0.6933 - val_f1_m: 0.6708
Epoch 40/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0434 - accuracy: 0.7383 - f1_m: 0.7240 - val_loss: 0.9502 - val_accuracy: 0.6960 - val_f1_m: 0.6859
Epoch 41/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0428 - accuracy: 0.7416 - f1_m: 0.7303 - val_loss: 0.8923 - val_accuracy: 0.7190 - val_f1_m: 0.7073
Epoch 42/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0394 - accuracy: 0.7613 - f1_m: 0.7522 - val_loss: 1.0069 - val_accuracy: 0.6720 - val_f1_m: 0.6601
Epoch 43/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0404 - accuracy: 0.7559 - f1_m: 0.7448 - val_loss: 0.8779 - val_accuracy: 0.7170 - val_f1_m: 0.7071
Epoch 44/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0376 - accuracy: 0.7713 - f1_m: 0.7641 - val_loss: 0.8671 - val_accuracy: 0.7253 - val_f1_m: 0.7197
Epoch 45/150
142/142 [==============================] - 2s 11ms/step - loss: 0.0350 - accuracy: 0.7900 - f1_m: 0.7839 - val_loss: 0.8621 - val_accuracy: 0.7293 - val_f1_m: 0.7284
Epoch 46/150
142/142 [==============================] - 2s 11ms/step - loss: 0.0337 - accuracy: 0.7975 - f1_m: 0.7861 - val_loss: 0.9861 - val_accuracy: 0.7007 - val_f1_m: 0.6970
Epoch 47/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0362 - accuracy: 0.7848 - f1_m: 0.7779 - val_loss: 0.7976 - val_accuracy: 0.7563 - val_f1_m: 0.7562
Epoch 48/150
142/142 [==============================] - 2s 11ms/step - loss: 0.0311 - accuracy: 0.8057 - f1_m: 0.8022 - val_loss: 0.9133 - val_accuracy: 0.7160 - val_f1_m: 0.7145
Epoch 49/150
142/142 [==============================] - 2s 11ms/step - loss: 0.0317 - accuracy: 0.8055 - f1_m: 0.8003 - val_loss: 0.8002 - val_accuracy: 0.7457 - val_f1_m: 0.7485
Epoch 50/150
142/142 [==============================] - 2s 11ms/step - loss: 0.0282 - accuracy: 0.8310 - f1_m: 0.8273 - val_loss: 0.7657 - val_accuracy: 0.7613 - val_f1_m: 0.7632
Epoch 51/150
142/142 [==============================] - 2s 10ms/step - loss: 0.0275 - accuracy: 0.8340 - f1_m: 0.8302 - val_loss: 0.7797 - val_accuracy: 0.7580 - val_f1_m: 0.7556
Epoch 52/150
142/142 [==============================] - 2s 10ms/step - loss: 0.0262 - accuracy: 0.8394 - f1_m: 0.8372 - val_loss: 0.7386 - val_accuracy: 0.7753 - val_f1_m: 0.7688
Epoch 53/150
142/142 [==============================] - 2s 10ms/step - loss: 0.0252 - accuracy: 0.8469 - f1_m: 0.8431 - val_loss: 0.7263 - val_accuracy: 0.7810 - val_f1_m: 0.7803
Epoch 54/150
142/142 [==============================] - 2s 11ms/step - loss: 0.0236 - accuracy: 0.8597 - f1_m: 0.8550 - val_loss: 0.7215 - val_accuracy: 0.7757 - val_f1_m: 0.7828
Epoch 55/150
142/142 [==============================] - 2s 12ms/step - loss: 0.0234 - accuracy: 0.8588 - f1_m: 0.8584 - val_loss: 0.8671 - val_accuracy: 0.7357 - val_f1_m: 0.7381
Epoch 56/150
142/142 [==============================] - 2s 10ms/step - loss: 0.0239 - accuracy: 0.8506 - f1_m: 0.8523 - val_loss: 0.9378 - val_accuracy: 0.7187 - val_f1_m: 0.7199
Epoch 57/150
142/142 [==============================] - 2s 11ms/step - loss: 0.0212 - accuracy: 0.8694 - f1_m: 0.8722 - val_loss: 0.7192 - val_accuracy: 0.7873 - val_f1_m: 0.7918
Epoch 58/150
142/142 [==============================] - 2s 12ms/step - loss: 0.0198 - accuracy: 0.8838 - f1_m: 0.8773 - val_loss: 0.9001 - val_accuracy: 0.7373 - val_f1_m: 0.7374
Epoch 59/150
142/142 [==============================] - 2s 11ms/step - loss: 0.0197 - accuracy: 0.8831 - f1_m: 0.8801 - val_loss: 0.7592 - val_accuracy: 0.7717 - val_f1_m: 0.7733
Epoch 60/150
142/142 [==============================] - 2s 11ms/step - loss: 0.0181 - accuracy: 0.8872 - f1_m: 0.8875 - val_loss: 0.7200 - val_accuracy: 0.7837 - val_f1_m: 0.7901
Epoch 61/150
142/142 [==============================] - 2s 12ms/step - loss: 0.0167 - accuracy: 0.8998 - f1_m: 0.8991 - val_loss: 0.7803 - val_accuracy: 0.7683 - val_f1_m: 0.7697
Epoch 62/150
142/142 [==============================] - 2s 11ms/step - loss: 0.0172 - accuracy: 0.8954 - f1_m: 0.8921 - val_loss: 0.7219 - val_accuracy: 0.7957 - val_f1_m: 0.7961
Epoch 63/150
142/142 [==============================] - 2s 10ms/step - loss: 0.0167 - accuracy: 0.8980 - f1_m: 0.8986 - val_loss: 0.6808 - val_accuracy: 0.8063 - val_f1_m: 0.8059
Epoch 64/150
142/142 [==============================] - 2s 11ms/step - loss: 0.0145 - accuracy: 0.9146 - f1_m: 0.9118 - val_loss: 0.7198 - val_accuracy: 0.8030 - val_f1_m: 0.8043
Epoch 65/150
142/142 [==============================] - 2s 10ms/step - loss: 0.0178 - accuracy: 0.8898 - f1_m: 0.8903 - val_loss: 0.6456 - val_accuracy: 0.8133 - val_f1_m: 0.8169
Epoch 66/150
142/142 [==============================] - 2s 11ms/step - loss: 0.0133 - accuracy: 0.9230 - f1_m: 0.9230 - val_loss: 0.6673 - val_accuracy: 0.8087 - val_f1_m: 0.8114
Epoch 67/150
142/142 [==============================] - 2s 11ms/step - loss: 0.0127 - accuracy: 0.9246 - f1_m: 0.9241 - val_loss: 0.6435 - val_accuracy: 0.8177 - val_f1_m: 0.8196
Epoch 68/150
142/142 [==============================] - 2s 11ms/step - loss: 0.0123 - accuracy: 0.9261 - f1_m: 0.9270 - val_loss: 0.7304 - val_accuracy: 0.7987 - val_f1_m: 0.8013
Epoch 69/150
142/142 [==============================] - 2s 10ms/step - loss: 0.0113 - accuracy: 0.9313 - f1_m: 0.9323 - val_loss: 0.6590 - val_accuracy: 0.8163 - val_f1_m: 0.8211
Epoch 70/150
142/142 [==============================] - 2s 10ms/step - loss: 0.0116 - accuracy: 0.9289 - f1_m: 0.9280 - val_loss: 0.6487 - val_accuracy: 0.8263 - val_f1_m: 0.8291
Epoch 71/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0111 - accuracy: 0.9293 - f1_m: 0.9307 - val_loss: 0.6739 - val_accuracy: 0.8107 - val_f1_m: 0.8130
Epoch 72/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0107 - accuracy: 0.9364 - f1_m: 0.9366 - val_loss: 0.6636 - val_accuracy: 0.8193 - val_f1_m: 0.8227
Epoch 73/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0098 - accuracy: 0.9428 - f1_m: 0.9425 - val_loss: 0.6506 - val_accuracy: 0.8260 - val_f1_m: 0.8325
Epoch 74/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0086 - accuracy: 0.9513 - f1_m: 0.9513 - val_loss: 0.6642 - val_accuracy: 0.8197 - val_f1_m: 0.8246
Epoch 75/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0080 - accuracy: 0.9536 - f1_m: 0.9532 - val_loss: 0.6997 - val_accuracy: 0.8173 - val_f1_m: 0.8211
Epoch 76/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0085 - accuracy: 0.9498 - f1_m: 0.9490 - val_loss: 0.6682 - val_accuracy: 0.8283 - val_f1_m: 0.8335
Epoch 77/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0072 - accuracy: 0.9577 - f1_m: 0.9574 - val_loss: 0.6829 - val_accuracy: 0.8190 - val_f1_m: 0.8245
Epoch 77: early stopping
CPU times: total: 1min 31s
Wall time: 2min 2s

%%time
BaseModel_128 =  create_Basemodel((128,128,1),learning_rate,momentum,class_weights_dict,dataset_train_128,dataset_test_128,steps_per_epoch)

Epoch 1/150
142/142 [==============================] - 6s 38ms/step - loss: 0.1501 - accuracy: 0.1097 - f1_m: 0.0000e+00 - val_loss: 2.6740 - val_accuracy: 0.1070 - val_f1_m: 0.0000e+00
Epoch 2/150
142/142 [==============================] - 5s 37ms/step - loss: 0.1462 - accuracy: 0.1369 - f1_m: 0.0000e+00 - val_loss: 2.5610 - val_accuracy: 0.1557 - val_f1_m: 0.0013
Epoch 3/150
142/142 [==============================] - 5s 37ms/step - loss: 0.1376 - accuracy: 0.2045 - f1_m: 0.0045 - val_loss: 2.3882 - val_accuracy: 0.2573 - val_f1_m: 0.0143
Epoch 4/150
142/142 [==============================] - 5s 37ms/step - loss: 0.1279 - accuracy: 0.2913 - f1_m: 0.0171 - val_loss: 2.2287 - val_accuracy: 0.3133 - val_f1_m: 0.0567
Epoch 5/150
142/142 [==============================] - 5s 37ms/step - loss: 0.1182 - accuracy: 0.3499 - f1_m: 0.0758 - val_loss: 2.0560 - val_accuracy: 0.3567 - val_f1_m: 0.1162
Epoch 6/150
142/142 [==============================] - 5s 37ms/step - loss: 0.1086 - accuracy: 0.4025 - f1_m: 0.1617 - val_loss: 1.9835 - val_accuracy: 0.3863 - val_f1_m: 0.2103
Epoch 7/150
142/142 [==============================] - 5s 37ms/step - loss: 0.0998 - accuracy: 0.4404 - f1_m: 0.2544 - val_loss: 1.6977 - val_accuracy: 0.4843 - val_f1_m: 0.3251
Epoch 8/150
142/142 [==============================] - 5s 37ms/step - loss: 0.0873 - accuracy: 0.5107 - f1_m: 0.3821 - val_loss: 1.5540 - val_accuracy: 0.5150 - val_f1_m: 0.4163
Epoch 9/150
142/142 [==============================] - 5s 38ms/step - loss: 0.0798 - accuracy: 0.5488 - f1_m: 0.4506 - val_loss: 1.4321 - val_accuracy: 0.5510 - val_f1_m: 0.4723
Epoch 10/150
142/142 [==============================] - 5s 37ms/step - loss: 0.0704 - accuracy: 0.6036 - f1_m: 0.5335 - val_loss: 1.5309 - val_accuracy: 0.5250 - val_f1_m: 0.4842
Epoch 11/150
142/142 [==============================] - 5s 37ms/step - loss: 0.0679 - accuracy: 0.6095 - f1_m: 0.5511 - val_loss: 1.3149 - val_accuracy: 0.5920 - val_f1_m: 0.5598
Epoch 12/150
142/142 [==============================] - 5s 37ms/step - loss: 0.0573 - accuracy: 0.6804 - f1_m: 0.6445 - val_loss: 1.2897 - val_accuracy: 0.6047 - val_f1_m: 0.5782
Epoch 13/150
142/142 [==============================] - 5s 37ms/step - loss: 0.0508 - accuracy: 0.7076 - f1_m: 0.6814 - val_loss: 1.0497 - val_accuracy: 0.6750 - val_f1_m: 0.6606
Epoch 14/150
142/142 [==============================] - 5s 37ms/step - loss: 0.0439 - accuracy: 0.7496 - f1_m: 0.7361 - val_loss: 1.0151 - val_accuracy: 0.6743 - val_f1_m: 0.6627
Epoch 15/150
142/142 [==============================] - 5s 37ms/step - loss: 0.0364 - accuracy: 0.7884 - f1_m: 0.7787 - val_loss: 1.0565 - val_accuracy: 0.6787 - val_f1_m: 0.6762
Epoch 16/150
142/142 [==============================] - 5s 37ms/step - loss: 0.0335 - accuracy: 0.8063 - f1_m: 0.8004 - val_loss: 0.9942 - val_accuracy: 0.6933 - val_f1_m: 0.6880
Epoch 17/150
142/142 [==============================] - 5s 37ms/step - loss: 0.0272 - accuracy: 0.8444 - f1_m: 0.8407 - val_loss: 0.8720 - val_accuracy: 0.7370 - val_f1_m: 0.7375
Epoch 18/150
142/142 [==============================] - 5s 37ms/step - loss: 0.0254 - accuracy: 0.8520 - f1_m: 0.8480 - val_loss: 0.9285 - val_accuracy: 0.7283 - val_f1_m: 0.7267
Epoch 19/150
142/142 [==============================] - 5s 37ms/step - loss: 0.0221 - accuracy: 0.8705 - f1_m: 0.8708 - val_loss: 0.8814 - val_accuracy: 0.7360 - val_f1_m: 0.7415
Epoch 20/150
142/142 [==============================] - 5s 38ms/step - loss: 0.0168 - accuracy: 0.9010 - f1_m: 0.9012 - val_loss: 0.9298 - val_accuracy: 0.7370 - val_f1_m: 0.7399
Epoch 21/150
142/142 [==============================] - 5s 38ms/step - loss: 0.0148 - accuracy: 0.9156 - f1_m: 0.9150 - val_loss: 0.9457 - val_accuracy: 0.7377 - val_f1_m: 0.7400
Epoch 22/150
142/142 [==============================] - 6s 38ms/step - loss: 0.0128 - accuracy: 0.9277 - f1_m: 0.9276 - val_loss: 0.9230 - val_accuracy: 0.7470 - val_f1_m: 0.7510
Epoch 23/150
142/142 [==============================] - 5s 38ms/step - loss: 0.0102 - accuracy: 0.9448 - f1_m: 0.9445 - val_loss: 0.8486 - val_accuracy: 0.7767 - val_f1_m: 0.7809
Epoch 24/150
142/142 [==============================] - 6s 38ms/step - loss: 0.0078 - accuracy: 0.9589 - f1_m: 0.9592 - val_loss: 0.8939 - val_accuracy: 0.7603 - val_f1_m: 0.7691
Epoch 25/150
142/142 [==============================] - 5s 38ms/step - loss: 0.0066 - accuracy: 0.9684 - f1_m: 0.9676 - val_loss: 0.9343 - val_accuracy: 0.7583 - val_f1_m: 0.7621
Epoch 26/150
142/142 [==============================] - 5s 37ms/step - loss: 0.0054 - accuracy: 0.9765 - f1_m: 0.9765 - val_loss: 0.9103 - val_accuracy: 0.7797 - val_f1_m: 0.7794
Epoch 27/150
142/142 [==============================] - 5s 37ms/step - loss: 0.0043 - accuracy: 0.9822 - f1_m: 0.9818 - val_loss: 0.9319 - val_accuracy: 0.7710 - val_f1_m: 0.7763
Epoch 28/150
142/142 [==============================] - 5s 37ms/step - loss: 0.0034 - accuracy: 0.9878 - f1_m: 0.9872 - val_loss: 1.0080 - val_accuracy: 0.7583 - val_f1_m: 0.7614
Epoch 29/150
142/142 [==============================] - 6s 39ms/step - loss: 0.0027 - accuracy: 0.9888 - f1_m: 0.9884 - val_loss: 0.9778 - val_accuracy: 0.7640 - val_f1_m: 0.7708
Epoch 30/150
142/142 [==============================] - 5s 37ms/step - loss: 0.0019 - accuracy: 0.9951 - f1_m: 0.9946 - val_loss: 0.9594 - val_accuracy: 0.7840 - val_f1_m: 0.7898
Epoch 31/150
142/142 [==============================] - 5s 38ms/step - loss: 0.0022 - accuracy: 0.9929 - f1_m: 0.9927 - val_loss: 1.0206 - val_accuracy: 0.7727 - val_f1_m: 0.7766
Epoch 32/150
142/142 [==============================] - 5s 38ms/step - loss: 0.0015 - accuracy: 0.9958 - f1_m: 0.9959 - val_loss: 0.9812 - val_accuracy: 0.7863 - val_f1_m: 0.7894
Epoch 33/150
142/142 [==============================] - 6s 41ms/step - loss: 0.0017 - accuracy: 0.9950 - f1_m: 0.9953 - val_loss: 1.1993 - val_accuracy: 0.7537 - val_f1_m: 0.7558
Epoch 33: early stopping
CPU times: total: 1min 20s
Wall time: 2min 59s

BaseModel_128.summary()
BaseModel_31.summary()

Model: "sequential"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 rescaling (Rescaling)       (None, 128, 128, 1)       0         
                                                                 
 conv2d (Conv2D)             (None, 126, 126, 64)      640       
                                                                 
 max_pooling2d (MaxPooling2D  (None, 63, 63, 64)       0         
 )                                                               
                                                                 
 conv2d_1 (Conv2D)           (None, 61, 61, 64)        36928     
                                                                 
 max_pooling2d_1 (MaxPooling  (None, 30, 30, 64)       0         
 2D)                                                             
                                                                 
 flatten (Flatten)           (None, 57600)             0         
                                                                 
 dense (Dense)               (None, 128)               7372928   
                                                                 
 dense_1 (Dense)             (None, 128)               16512     
                                                                 
 dense_2 (Dense)             (None, 15)                1935      
                                                                 
=================================================================
Total params: 7,428,943
Trainable params: 7,428,943
Non-trainable params: 0
_________________________________________________________________
Model: "sequential"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 rescaling (Rescaling)       (None, 31, 31, 1)         0         
                                                                 
 conv2d (Conv2D)             (None, 29, 29, 64)        640       
                                                                 
 max_pooling2d (MaxPooling2D  (None, 14, 14, 64)       0         
 )                                                               
                                                                 
 conv2d_1 (Conv2D)           (None, 12, 12, 64)        36928     
                                                                 
 max_pooling2d_1 (MaxPooling  (None, 6, 6, 64)         0         
 2D)                                                             
                                                                 
 flatten (Flatten)           (None, 2304)              0         
                                                                 
 dense (Dense)               (None, 128)               295040    
                                                                 
 dense_1 (Dense)             (None, 128)               16512     
                                                                 
 dense_2 (Dense)             (None, 15)                1935      
                                                                 
=================================================================
Total params: 351,055
Trainable params: 351,055
Non-trainable params: 0
_________________________________________________________________

Observations:

• We can see that the bigger the input size greatly increases the number of parameters we need to train, thus increasing the training time and likely hood of overfitting in the bigger input size.

# evaluate_and_save_model(model, model.history, 'BaseModel', 64)
evaluate_and_save_model(BaseModel_128, 'BaseModel_128', 64,save_models=False)
evaluate_and_save_model(BaseModel_31, 'BaseModel_31', 64,save_models=False)

Model Name            BaseModel_128
Epochs                           33
Batch Size                       64
Train Loss                 0.001543
Val Loss                   0.981181
Train Acc                  0.995791
Val Acc                    0.786333
[Train - Val] Acc          0.209458
[Train - Val] Loss        -0.979638
Name: BaseModel_128, dtype: object
Model Name            BaseModel_31
Epochs                          77
Batch Size                      64
Train Loss                0.008457
Val Loss                  0.668227
Train Acc                 0.949823
Val Acc                   0.828333
[Train - Val] Acc         0.121489
[Train - Val] Loss        -0.65977
Name: BaseModel_31, dtype: object

baseline_128_history = json.load(open('./models/BaseModel_128_history.json', 'r'))
baseline_31_history = json.load(open('./models/BaseModel_31_history.json', 'r'))

compare_curves(baseline_31_history, baseline_128_history, '31x31', '128x128')

Obersevations:

• We can see as when the model increases in epochs, it becomes more generalise and our loss functions starts decreasing as well. However we can see that our model starts to overfit around the 30 epoch mark. We will be comparing and utilising the different overfiting prevention stratergiers latter on in the project.
• As expected , our 128x128 model overfitts alot more and alot faster, thus stopping relatively early during training.

Training with Data Augmentation

We will now train our baseline model on the dataset that we have augmented

%%time
BaseModel_128_DA = create_Basemodel((128,128,1),learning_rate,momentum,class_weights_dict,train_generator_128,dataset_test_128,steps_per_epoch)

Epoch 1/150
142/142 [==============================] - 6s 39ms/step - loss: 0.1504 - accuracy: 0.1002 - f1_m: 0.0000e+00 - val_loss: 2.6894 - val_accuracy: 0.1050 - val_f1_m: 0.0000e+00
Epoch 2/150
142/142 [==============================] - 5s 37ms/step - loss: 0.1484 - accuracy: 0.1263 - f1_m: 0.0000e+00 - val_loss: 2.6209 - val_accuracy: 0.1107 - val_f1_m: 0.0000e+00
Epoch 3/150
142/142 [==============================] - 5s 38ms/step - loss: 0.1431 - accuracy: 0.1542 - f1_m: 0.0000e+00 - val_loss: 2.5045 - val_accuracy: 0.1923 - val_f1_m: 0.0000e+00
Epoch 4/150
142/142 [==============================] - 5s 37ms/step - loss: 0.1376 - accuracy: 0.2087 - f1_m: 0.0022 - val_loss: 2.3925 - val_accuracy: 0.2300 - val_f1_m: 0.0141
Epoch 5/150
142/142 [==============================] - 6s 44ms/step - loss: 0.1306 - accuracy: 0.2807 - f1_m: 0.0109 - val_loss: 2.2512 - val_accuracy: 0.2987 - val_f1_m: 0.0596
Epoch 6/150
142/142 [==============================] - 6s 38ms/step - loss: 0.1244 - accuracy: 0.3097 - f1_m: 0.0316 - val_loss: 2.1014 - val_accuracy: 0.3280 - val_f1_m: 0.0901
Epoch 7/150
142/142 [==============================] - 5s 37ms/step - loss: 0.1165 - accuracy: 0.3501 - f1_m: 0.0778 - val_loss: 2.0123 - val_accuracy: 0.3537 - val_f1_m: 0.1620
Epoch 8/150
142/142 [==============================] - 5s 37ms/step - loss: 0.1090 - accuracy: 0.3769 - f1_m: 0.1493 - val_loss: 1.8024 - val_accuracy: 0.4157 - val_f1_m: 0.2663
Epoch 9/150
142/142 [==============================] - 5s 37ms/step - loss: 0.0995 - accuracy: 0.4322 - f1_m: 0.2444 - val_loss: 1.6714 - val_accuracy: 0.4643 - val_f1_m: 0.3240
Epoch 10/150
142/142 [==============================] - 5s 37ms/step - loss: 0.0912 - accuracy: 0.4750 - f1_m: 0.3230 - val_loss: 1.5203 - val_accuracy: 0.5193 - val_f1_m: 0.4070
Epoch 11/150
142/142 [==============================] - 5s 37ms/step - loss: 0.0843 - accuracy: 0.5079 - f1_m: 0.4080 - val_loss: 1.4304 - val_accuracy: 0.5500 - val_f1_m: 0.4468
Epoch 12/150
142/142 [==============================] - 5s 37ms/step - loss: 0.0779 - accuracy: 0.5544 - f1_m: 0.4647 - val_loss: 1.4116 - val_accuracy: 0.5510 - val_f1_m: 0.4897
Epoch 13/150
142/142 [==============================] - 5s 38ms/step - loss: 0.0728 - accuracy: 0.5829 - f1_m: 0.5198 - val_loss: 1.2663 - val_accuracy: 0.5933 - val_f1_m: 0.5450
Epoch 14/150
142/142 [==============================] - 5s 37ms/step - loss: 0.0660 - accuracy: 0.6206 - f1_m: 0.5715 - val_loss: 1.0937 - val_accuracy: 0.6513 - val_f1_m: 0.6162
Epoch 15/150
142/142 [==============================] - 5s 37ms/step - loss: 0.0659 - accuracy: 0.6314 - f1_m: 0.5842 - val_loss: 1.2162 - val_accuracy: 0.6163 - val_f1_m: 0.5645
Epoch 16/150
142/142 [==============================] - 5s 37ms/step - loss: 0.0567 - accuracy: 0.6804 - f1_m: 0.6487 - val_loss: 1.0605 - val_accuracy: 0.6640 - val_f1_m: 0.6474
Epoch 17/150
142/142 [==============================] - 5s 37ms/step - loss: 0.0557 - accuracy: 0.6808 - f1_m: 0.6565 - val_loss: 1.1775 - val_accuracy: 0.6270 - val_f1_m: 0.6025
Epoch 18/150
142/142 [==============================] - 5s 37ms/step - loss: 0.0513 - accuracy: 0.7134 - f1_m: 0.6940 - val_loss: 1.2015 - val_accuracy: 0.6107 - val_f1_m: 0.6081
Epoch 19/150
142/142 [==============================] - 5s 36ms/step - loss: 0.0480 - accuracy: 0.7288 - f1_m: 0.7120 - val_loss: 0.9158 - val_accuracy: 0.7107 - val_f1_m: 0.6936
Epoch 20/150
142/142 [==============================] - 5s 36ms/step - loss: 0.0457 - accuracy: 0.7376 - f1_m: 0.7229 - val_loss: 0.8654 - val_accuracy: 0.7343 - val_f1_m: 0.7226
Epoch 21/150
142/142 [==============================] - 5s 36ms/step - loss: 0.0439 - accuracy: 0.7560 - f1_m: 0.7488 - val_loss: 0.8005 - val_accuracy: 0.7553 - val_f1_m: 0.7490
Epoch 22/150
142/142 [==============================] - 5s 36ms/step - loss: 0.0377 - accuracy: 0.7925 - f1_m: 0.7872 - val_loss: 0.8373 - val_accuracy: 0.7453 - val_f1_m: 0.7322
Epoch 23/150
142/142 [==============================] - 5s 36ms/step - loss: 0.0358 - accuracy: 0.7996 - f1_m: 0.7956 - val_loss: 0.8612 - val_accuracy: 0.7290 - val_f1_m: 0.7269
Epoch 24/150
142/142 [==============================] - 5s 37ms/step - loss: 0.0347 - accuracy: 0.8077 - f1_m: 0.8043 - val_loss: 0.7286 - val_accuracy: 0.7793 - val_f1_m: 0.7689
Epoch 25/150
142/142 [==============================] - 5s 36ms/step - loss: 0.0317 - accuracy: 0.8300 - f1_m: 0.8243 - val_loss: 0.7566 - val_accuracy: 0.7707 - val_f1_m: 0.7686
Epoch 26/150
142/142 [==============================] - 5s 37ms/step - loss: 0.0291 - accuracy: 0.8392 - f1_m: 0.8354 - val_loss: 1.0943 - val_accuracy: 0.6680 - val_f1_m: 0.6647
Epoch 27/150
142/142 [==============================] - 5s 37ms/step - loss: 0.0298 - accuracy: 0.8403 - f1_m: 0.8398 - val_loss: 0.7092 - val_accuracy: 0.7830 - val_f1_m: 0.7797
Epoch 28/150
142/142 [==============================] - 5s 37ms/step - loss: 0.0257 - accuracy: 0.8632 - f1_m: 0.8612 - val_loss: 0.7285 - val_accuracy: 0.7823 - val_f1_m: 0.7814
Epoch 29/150
142/142 [==============================] - 5s 37ms/step - loss: 0.0244 - accuracy: 0.8720 - f1_m: 0.8712 - val_loss: 0.6764 - val_accuracy: 0.7953 - val_f1_m: 0.7906
Epoch 30/150
142/142 [==============================] - 5s 37ms/step - loss: 0.0232 - accuracy: 0.8762 - f1_m: 0.8718 - val_loss: 0.7543 - val_accuracy: 0.7740 - val_f1_m: 0.7761
Epoch 31/150
142/142 [==============================] - 5s 37ms/step - loss: 0.0225 - accuracy: 0.8811 - f1_m: 0.8801 - val_loss: 0.6405 - val_accuracy: 0.8127 - val_f1_m: 0.8153
Epoch 32/150
142/142 [==============================] - 5s 36ms/step - loss: 0.0214 - accuracy: 0.8872 - f1_m: 0.8836 - val_loss: 0.6882 - val_accuracy: 0.7950 - val_f1_m: 0.7968
Epoch 33/150
142/142 [==============================] - 5s 36ms/step - loss: 0.0251 - accuracy: 0.8679 - f1_m: 0.8650 - val_loss: 0.6519 - val_accuracy: 0.7997 - val_f1_m: 0.8110
Epoch 34/150
142/142 [==============================] - 5s 37ms/step - loss: 0.0200 - accuracy: 0.8910 - f1_m: 0.8920 - val_loss: 0.7155 - val_accuracy: 0.7843 - val_f1_m: 0.7836
Epoch 35/150
142/142 [==============================] - 5s 37ms/step - loss: 0.0174 - accuracy: 0.9088 - f1_m: 0.9101 - val_loss: 0.6304 - val_accuracy: 0.8163 - val_f1_m: 0.8172
Epoch 36/150
142/142 [==============================] - 5s 37ms/step - loss: 0.0172 - accuracy: 0.9075 - f1_m: 0.9087 - val_loss: 0.6263 - val_accuracy: 0.8110 - val_f1_m: 0.8169
Epoch 37/150
142/142 [==============================] - 5s 37ms/step - loss: 0.0174 - accuracy: 0.9136 - f1_m: 0.9136 - val_loss: 0.6344 - val_accuracy: 0.8100 - val_f1_m: 0.8082
Epoch 38/150
142/142 [==============================] - 5s 36ms/step - loss: 0.0148 - accuracy: 0.9231 - f1_m: 0.9242 - val_loss: 0.6647 - val_accuracy: 0.8013 - val_f1_m: 0.8066
Epoch 39/150
142/142 [==============================] - 5s 36ms/step - loss: 0.0139 - accuracy: 0.9324 - f1_m: 0.9296 - val_loss: 0.6273 - val_accuracy: 0.8203 - val_f1_m: 0.8219
Epoch 40/150
142/142 [==============================] - 5s 37ms/step - loss: 0.0150 - accuracy: 0.9216 - f1_m: 0.9212 - val_loss: 0.6279 - val_accuracy: 0.8200 - val_f1_m: 0.8208
Epoch 41/150
142/142 [==============================] - 5s 37ms/step - loss: 0.0140 - accuracy: 0.9303 - f1_m: 0.9323 - val_loss: 0.6270 - val_accuracy: 0.8217 - val_f1_m: 0.8230
Epoch 42/150
142/142 [==============================] - 5s 37ms/step - loss: 0.0127 - accuracy: 0.9350 - f1_m: 0.9376 - val_loss: 0.6090 - val_accuracy: 0.8307 - val_f1_m: 0.8297
Epoch 43/150
142/142 [==============================] - 5s 37ms/step - loss: 0.0126 - accuracy: 0.9344 - f1_m: 0.9367 - val_loss: 0.6030 - val_accuracy: 0.8243 - val_f1_m: 0.8305
Epoch 44/150
142/142 [==============================] - 5s 37ms/step - loss: 0.0121 - accuracy: 0.9413 - f1_m: 0.9430 - val_loss: 0.6134 - val_accuracy: 0.8290 - val_f1_m: 0.8276
Epoch 45/150
142/142 [==============================] - 5s 38ms/step - loss: 0.0114 - accuracy: 0.9436 - f1_m: 0.9458 - val_loss: 0.5909 - val_accuracy: 0.8353 - val_f1_m: 0.8370
Epoch 46/150
142/142 [==============================] - 5s 37ms/step - loss: 0.0132 - accuracy: 0.9325 - f1_m: 0.9340 - val_loss: 0.6309 - val_accuracy: 0.8217 - val_f1_m: 0.8251
Epoch 47/150
142/142 [==============================] - 5s 37ms/step - loss: 0.0114 - accuracy: 0.9428 - f1_m: 0.9437 - val_loss: 0.6017 - val_accuracy: 0.8303 - val_f1_m: 0.8328
Epoch 48/150
142/142 [==============================] - 5s 37ms/step - loss: 0.0097 - accuracy: 0.9517 - f1_m: 0.9530 - val_loss: 0.5916 - val_accuracy: 0.8383 - val_f1_m: 0.8412
Epoch 49/150
142/142 [==============================] - 5s 37ms/step - loss: 0.0096 - accuracy: 0.9504 - f1_m: 0.9523 - val_loss: 0.6007 - val_accuracy: 0.8333 - val_f1_m: 0.8374
Epoch 50/150
142/142 [==============================] - 5s 37ms/step - loss: 0.0085 - accuracy: 0.9558 - f1_m: 0.9559 - val_loss: 0.6444 - val_accuracy: 0.8247 - val_f1_m: 0.8257
Epoch 51/150
142/142 [==============================] - 5s 36ms/step - loss: 0.0085 - accuracy: 0.9562 - f1_m: 0.9572 - val_loss: 0.6209 - val_accuracy: 0.8243 - val_f1_m: 0.8266
Epoch 52/150
142/142 [==============================] - 5s 36ms/step - loss: 0.0083 - accuracy: 0.9575 - f1_m: 0.9591 - val_loss: 0.6027 - val_accuracy: 0.8420 - val_f1_m: 0.8464
Epoch 53/150
142/142 [==============================] - 5s 36ms/step - loss: 0.0093 - accuracy: 0.9547 - f1_m: 0.9555 - val_loss: 0.6065 - val_accuracy: 0.8333 - val_f1_m: 0.8398
Epoch 54/150
142/142 [==============================] - 5s 36ms/step - loss: 0.0069 - accuracy: 0.9663 - f1_m: 0.9676 - val_loss: 0.7177 - val_accuracy: 0.8070 - val_f1_m: 0.8123
Epoch 55/150
142/142 [==============================] - 5s 36ms/step - loss: 0.0074 - accuracy: 0.9633 - f1_m: 0.9641 - val_loss: 0.6570 - val_accuracy: 0.8240 - val_f1_m: 0.8265
Epoch 55: early stopping
CPU times: total: 2min 22s
Wall time: 4min 56s

%%time
BaseModel_31_DA = create_Basemodel((31,31,1),learning_rate,momentum,class_weights_dict,train_generator_31,dataset_test_31,steps_per_epoch)

Epoch 1/150
142/142 [==============================] - 2s 12ms/step - loss: 0.1508 - accuracy: 0.0793 - f1_m: 0.0000e+00 - val_loss: 2.7048 - val_accuracy: 0.0827 - val_f1_m: 0.0000e+00
Epoch 2/150
142/142 [==============================] - 2s 13ms/step - loss: 0.1506 - accuracy: 0.0969 - f1_m: 0.0000e+00 - val_loss: 2.7015 - val_accuracy: 0.1107 - val_f1_m: 0.0000e+00
Epoch 3/150
142/142 [==============================] - 2s 12ms/step - loss: 0.1504 - accuracy: 0.1211 - f1_m: 0.0000e+00 - val_loss: 2.6979 - val_accuracy: 0.1223 - val_f1_m: 0.0000e+00
Epoch 4/150
142/142 [==============================] - 2s 12ms/step - loss: 0.1502 - accuracy: 0.1382 - f1_m: 0.0000e+00 - val_loss: 2.6924 - val_accuracy: 0.1310 - val_f1_m: 0.0000e+00
Epoch 5/150
142/142 [==============================] - 2s 11ms/step - loss: 0.1498 - accuracy: 0.1539 - f1_m: 0.0000e+00 - val_loss: 2.6830 - val_accuracy: 0.1590 - val_f1_m: 0.0000e+00
Epoch 6/150
142/142 [==============================] - 1s 10ms/step - loss: 0.1492 - accuracy: 0.1513 - f1_m: 0.0000e+00 - val_loss: 2.6671 - val_accuracy: 0.1563 - val_f1_m: 0.0000e+00
Epoch 7/150
142/142 [==============================] - 1s 10ms/step - loss: 0.1481 - accuracy: 0.1601 - f1_m: 0.0000e+00 - val_loss: 2.6405 - val_accuracy: 0.1490 - val_f1_m: 0.0000e+00
Epoch 8/150
142/142 [==============================] - 1s 10ms/step - loss: 0.1462 - accuracy: 0.1696 - f1_m: 0.0000e+00 - val_loss: 2.5919 - val_accuracy: 0.1670 - val_f1_m: 0.0000e+00
Epoch 9/150
142/142 [==============================] - 1s 10ms/step - loss: 0.1429 - accuracy: 0.1868 - f1_m: 0.0000e+00 - val_loss: 2.5146 - val_accuracy: 0.1967 - val_f1_m: 0.0000e+00
Epoch 10/150
142/142 [==============================] - 1s 10ms/step - loss: 0.1391 - accuracy: 0.1975 - f1_m: 4.3337e-04 - val_loss: 2.4319 - val_accuracy: 0.2197 - val_f1_m: 0.0033
Epoch 11/150
142/142 [==============================] - 1s 10ms/step - loss: 0.1353 - accuracy: 0.2228 - f1_m: 0.0039 - val_loss: 2.3982 - val_accuracy: 0.2200 - val_f1_m: 0.0033
Epoch 12/150
142/142 [==============================] - 1s 10ms/step - loss: 0.1331 - accuracy: 0.2390 - f1_m: 0.0065 - val_loss: 2.3215 - val_accuracy: 0.2597 - val_f1_m: 0.0129
Epoch 13/150
142/142 [==============================] - 1s 10ms/step - loss: 0.1307 - accuracy: 0.2479 - f1_m: 0.0077 - val_loss: 2.2925 - val_accuracy: 0.2567 - val_f1_m: 0.0181
Epoch 14/150
142/142 [==============================] - 1s 10ms/step - loss: 0.1288 - accuracy: 0.2626 - f1_m: 0.0107 - val_loss: 2.2521 - val_accuracy: 0.2903 - val_f1_m: 0.0193
Epoch 15/150
142/142 [==============================] - 1s 10ms/step - loss: 0.1268 - accuracy: 0.2727 - f1_m: 0.0105 - val_loss: 2.2161 - val_accuracy: 0.2930 - val_f1_m: 0.0301
Epoch 16/150
142/142 [==============================] - 1s 10ms/step - loss: 0.1245 - accuracy: 0.2913 - f1_m: 0.0133 - val_loss: 2.1734 - val_accuracy: 0.2943 - val_f1_m: 0.0388
Epoch 17/150
142/142 [==============================] - 1s 10ms/step - loss: 0.1224 - accuracy: 0.3070 - f1_m: 0.0159 - val_loss: 2.1592 - val_accuracy: 0.3143 - val_f1_m: 0.0537
Epoch 18/150
142/142 [==============================] - 1s 10ms/step - loss: 0.1195 - accuracy: 0.3144 - f1_m: 0.0256 - val_loss: 2.0639 - val_accuracy: 0.3213 - val_f1_m: 0.0712
Epoch 19/150
142/142 [==============================] - 1s 10ms/step - loss: 0.1153 - accuracy: 0.3427 - f1_m: 0.0444 - val_loss: 2.0013 - val_accuracy: 0.3580 - val_f1_m: 0.0651
Epoch 20/150
142/142 [==============================] - 1s 10ms/step - loss: 0.1124 - accuracy: 0.3611 - f1_m: 0.0634 - val_loss: 1.9307 - val_accuracy: 0.3773 - val_f1_m: 0.1251
Epoch 21/150
142/142 [==============================] - 1s 10ms/step - loss: 0.1080 - accuracy: 0.3809 - f1_m: 0.1031 - val_loss: 1.9014 - val_accuracy: 0.3747 - val_f1_m: 0.1666
Epoch 22/150
142/142 [==============================] - 1s 10ms/step - loss: 0.1039 - accuracy: 0.4014 - f1_m: 0.1482 - val_loss: 1.7614 - val_accuracy: 0.4287 - val_f1_m: 0.2016
Epoch 23/150
142/142 [==============================] - 2s 10ms/step - loss: 0.1003 - accuracy: 0.4181 - f1_m: 0.1787 - val_loss: 1.6786 - val_accuracy: 0.4643 - val_f1_m: 0.2329
Epoch 24/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0973 - accuracy: 0.4366 - f1_m: 0.2212 - val_loss: 1.6230 - val_accuracy: 0.4830 - val_f1_m: 0.2731
Epoch 25/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0939 - accuracy: 0.4489 - f1_m: 0.2622 - val_loss: 1.5673 - val_accuracy: 0.4923 - val_f1_m: 0.3194
Epoch 26/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0894 - accuracy: 0.4776 - f1_m: 0.3120 - val_loss: 1.5119 - val_accuracy: 0.5077 - val_f1_m: 0.3641
Epoch 27/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0878 - accuracy: 0.4899 - f1_m: 0.3417 - val_loss: 1.4682 - val_accuracy: 0.5330 - val_f1_m: 0.4041
Epoch 28/150
142/142 [==============================] - 1s 9ms/step - loss: 0.0834 - accuracy: 0.5123 - f1_m: 0.3817 - val_loss: 1.8727 - val_accuracy: 0.3927 - val_f1_m: 0.3151
Epoch 29/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0827 - accuracy: 0.5062 - f1_m: 0.3943 - val_loss: 1.6014 - val_accuracy: 0.4813 - val_f1_m: 0.3869
Epoch 30/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0796 - accuracy: 0.5284 - f1_m: 0.4210 - val_loss: 1.3489 - val_accuracy: 0.5577 - val_f1_m: 0.4683
Epoch 31/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0775 - accuracy: 0.5407 - f1_m: 0.4477 - val_loss: 1.4562 - val_accuracy: 0.5323 - val_f1_m: 0.4606
Epoch 32/150
142/142 [==============================] - 1s 9ms/step - loss: 0.0761 - accuracy: 0.5436 - f1_m: 0.4566 - val_loss: 1.3154 - val_accuracy: 0.5783 - val_f1_m: 0.5007
Epoch 33/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0737 - accuracy: 0.5629 - f1_m: 0.4820 - val_loss: 1.2723 - val_accuracy: 0.5853 - val_f1_m: 0.5270
Epoch 34/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0702 - accuracy: 0.5860 - f1_m: 0.5142 - val_loss: 1.1762 - val_accuracy: 0.6250 - val_f1_m: 0.5649
Epoch 35/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0688 - accuracy: 0.5908 - f1_m: 0.5268 - val_loss: 1.6810 - val_accuracy: 0.4637 - val_f1_m: 0.4192
Epoch 36/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0665 - accuracy: 0.6061 - f1_m: 0.5435 - val_loss: 1.2759 - val_accuracy: 0.5880 - val_f1_m: 0.5301
Epoch 37/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0652 - accuracy: 0.6138 - f1_m: 0.5557 - val_loss: 1.2535 - val_accuracy: 0.6020 - val_f1_m: 0.5501
Epoch 38/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0635 - accuracy: 0.6258 - f1_m: 0.5729 - val_loss: 1.1106 - val_accuracy: 0.6397 - val_f1_m: 0.6023
Epoch 39/150
142/142 [==============================] - 2s 11ms/step - loss: 0.0610 - accuracy: 0.6402 - f1_m: 0.5897 - val_loss: 1.1760 - val_accuracy: 0.6217 - val_f1_m: 0.5791
Epoch 40/150
142/142 [==============================] - 2s 10ms/step - loss: 0.0598 - accuracy: 0.6480 - f1_m: 0.6143 - val_loss: 1.0238 - val_accuracy: 0.6710 - val_f1_m: 0.6394
Epoch 41/150
142/142 [==============================] - 2s 11ms/step - loss: 0.0576 - accuracy: 0.6619 - f1_m: 0.6215 - val_loss: 1.3931 - val_accuracy: 0.5470 - val_f1_m: 0.5096
Epoch 42/150
142/142 [==============================] - 2s 11ms/step - loss: 0.0577 - accuracy: 0.6580 - f1_m: 0.6260 - val_loss: 1.0681 - val_accuracy: 0.6627 - val_f1_m: 0.6250
Epoch 43/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0548 - accuracy: 0.6745 - f1_m: 0.6432 - val_loss: 0.9726 - val_accuracy: 0.6923 - val_f1_m: 0.6661
Epoch 44/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0532 - accuracy: 0.6891 - f1_m: 0.6568 - val_loss: 0.8974 - val_accuracy: 0.7193 - val_f1_m: 0.6964
Epoch 45/150
142/142 [==============================] - 2s 10ms/step - loss: 0.0512 - accuracy: 0.7018 - f1_m: 0.6750 - val_loss: 0.8941 - val_accuracy: 0.7227 - val_f1_m: 0.7013
Epoch 46/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0509 - accuracy: 0.7005 - f1_m: 0.6785 - val_loss: 1.0210 - val_accuracy: 0.6767 - val_f1_m: 0.6593
Epoch 47/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0515 - accuracy: 0.6976 - f1_m: 0.6680 - val_loss: 0.9996 - val_accuracy: 0.6800 - val_f1_m: 0.6668
Epoch 48/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0486 - accuracy: 0.7137 - f1_m: 0.6931 - val_loss: 0.8632 - val_accuracy: 0.7227 - val_f1_m: 0.7114
Epoch 49/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0466 - accuracy: 0.7342 - f1_m: 0.7098 - val_loss: 0.9263 - val_accuracy: 0.7050 - val_f1_m: 0.6927
Epoch 50/150
142/142 [==============================] - 2s 11ms/step - loss: 0.0460 - accuracy: 0.7286 - f1_m: 0.7066 - val_loss: 0.8345 - val_accuracy: 0.7360 - val_f1_m: 0.7204
Epoch 51/150
142/142 [==============================] - 2s 11ms/step - loss: 0.0456 - accuracy: 0.7295 - f1_m: 0.7110 - val_loss: 1.2324 - val_accuracy: 0.6007 - val_f1_m: 0.5911
Epoch 52/150
142/142 [==============================] - 2s 11ms/step - loss: 0.0452 - accuracy: 0.7324 - f1_m: 0.7163 - val_loss: 0.7865 - val_accuracy: 0.7553 - val_f1_m: 0.7440
Epoch 53/150
142/142 [==============================] - 2s 11ms/step - loss: 0.0425 - accuracy: 0.7486 - f1_m: 0.7385 - val_loss: 0.9750 - val_accuracy: 0.6930 - val_f1_m: 0.6866
Epoch 54/150
142/142 [==============================] - 2s 10ms/step - loss: 0.0435 - accuracy: 0.7386 - f1_m: 0.7262 - val_loss: 0.8523 - val_accuracy: 0.7400 - val_f1_m: 0.7274
Epoch 55/150
142/142 [==============================] - 2s 11ms/step - loss: 0.0412 - accuracy: 0.7579 - f1_m: 0.7453 - val_loss: 0.7668 - val_accuracy: 0.7530 - val_f1_m: 0.7473
Epoch 56/150
142/142 [==============================] - 2s 11ms/step - loss: 0.0391 - accuracy: 0.7712 - f1_m: 0.7651 - val_loss: 0.8249 - val_accuracy: 0.7290 - val_f1_m: 0.7329
Epoch 57/150
142/142 [==============================] - 2s 10ms/step - loss: 0.0389 - accuracy: 0.7764 - f1_m: 0.7681 - val_loss: 0.7121 - val_accuracy: 0.7803 - val_f1_m: 0.7702
Epoch 58/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0398 - accuracy: 0.7660 - f1_m: 0.7536 - val_loss: 0.8234 - val_accuracy: 0.7367 - val_f1_m: 0.7361
Epoch 59/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0394 - accuracy: 0.7720 - f1_m: 0.7595 - val_loss: 0.7429 - val_accuracy: 0.7620 - val_f1_m: 0.7556
Epoch 60/150
142/142 [==============================] - 2s 10ms/step - loss: 0.0364 - accuracy: 0.7899 - f1_m: 0.7827 - val_loss: 0.7232 - val_accuracy: 0.7727 - val_f1_m: 0.7674
Epoch 61/150
142/142 [==============================] - 2s 11ms/step - loss: 0.0352 - accuracy: 0.7920 - f1_m: 0.7818 - val_loss: 0.6849 - val_accuracy: 0.7827 - val_f1_m: 0.7763
Epoch 62/150
142/142 [==============================] - 2s 10ms/step - loss: 0.0353 - accuracy: 0.7954 - f1_m: 0.7898 - val_loss: 0.7853 - val_accuracy: 0.7533 - val_f1_m: 0.7467
Epoch 63/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0353 - accuracy: 0.7912 - f1_m: 0.7862 - val_loss: 0.6773 - val_accuracy: 0.7917 - val_f1_m: 0.7869
Epoch 64/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0338 - accuracy: 0.8023 - f1_m: 0.7959 - val_loss: 0.6696 - val_accuracy: 0.7910 - val_f1_m: 0.7843
Epoch 65/150
142/142 [==============================] - 2s 10ms/step - loss: 0.0341 - accuracy: 0.8043 - f1_m: 0.7933 - val_loss: 0.7280 - val_accuracy: 0.7637 - val_f1_m: 0.7614
Epoch 66/150
142/142 [==============================] - 2s 10ms/step - loss: 0.0328 - accuracy: 0.8105 - f1_m: 0.8039 - val_loss: 0.6342 - val_accuracy: 0.8083 - val_f1_m: 0.8008
Epoch 67/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0312 - accuracy: 0.8179 - f1_m: 0.8146 - val_loss: 0.6580 - val_accuracy: 0.7893 - val_f1_m: 0.7898
Epoch 68/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0298 - accuracy: 0.8284 - f1_m: 0.8226 - val_loss: 0.6354 - val_accuracy: 0.7963 - val_f1_m: 0.7984
Epoch 69/150
142/142 [==============================] - 2s 10ms/step - loss: 0.0314 - accuracy: 0.8172 - f1_m: 0.8141 - val_loss: 0.6658 - val_accuracy: 0.7957 - val_f1_m: 0.7931
Epoch 70/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0298 - accuracy: 0.8262 - f1_m: 0.8215 - val_loss: 0.6169 - val_accuracy: 0.8057 - val_f1_m: 0.8048
Epoch 71/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0293 - accuracy: 0.8324 - f1_m: 0.8301 - val_loss: 0.6590 - val_accuracy: 0.7980 - val_f1_m: 0.7968
Epoch 72/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0298 - accuracy: 0.8265 - f1_m: 0.8250 - val_loss: 0.7041 - val_accuracy: 0.7757 - val_f1_m: 0.7759
Epoch 73/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0274 - accuracy: 0.8415 - f1_m: 0.8401 - val_loss: 0.6140 - val_accuracy: 0.8087 - val_f1_m: 0.8096
Epoch 74/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0280 - accuracy: 0.8367 - f1_m: 0.8330 - val_loss: 0.6269 - val_accuracy: 0.8050 - val_f1_m: 0.8046
Epoch 75/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0260 - accuracy: 0.8516 - f1_m: 0.8516 - val_loss: 0.6583 - val_accuracy: 0.7990 - val_f1_m: 0.8003
Epoch 76/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0238 - accuracy: 0.8623 - f1_m: 0.8588 - val_loss: 0.6901 - val_accuracy: 0.7887 - val_f1_m: 0.7844
Epoch 77/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0263 - accuracy: 0.8485 - f1_m: 0.8442 - val_loss: 0.6238 - val_accuracy: 0.8107 - val_f1_m: 0.8099
Epoch 78/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0251 - accuracy: 0.8593 - f1_m: 0.8548 - val_loss: 0.5692 - val_accuracy: 0.8227 - val_f1_m: 0.8219
Epoch 79/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0237 - accuracy: 0.8683 - f1_m: 0.8659 - val_loss: 0.6480 - val_accuracy: 0.8087 - val_f1_m: 0.8068
Epoch 80/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0236 - accuracy: 0.8639 - f1_m: 0.8647 - val_loss: 0.6324 - val_accuracy: 0.8027 - val_f1_m: 0.8020
Epoch 81/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0234 - accuracy: 0.8623 - f1_m: 0.8621 - val_loss: 0.6136 - val_accuracy: 0.8100 - val_f1_m: 0.8134
Epoch 82/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0237 - accuracy: 0.8670 - f1_m: 0.8661 - val_loss: 0.6808 - val_accuracy: 0.7907 - val_f1_m: 0.7936
Epoch 83/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0230 - accuracy: 0.8694 - f1_m: 0.8654 - val_loss: 0.6523 - val_accuracy: 0.8033 - val_f1_m: 0.8031
Epoch 84/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0236 - accuracy: 0.8630 - f1_m: 0.8620 - val_loss: 0.5474 - val_accuracy: 0.8283 - val_f1_m: 0.8350
Epoch 85/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0207 - accuracy: 0.8775 - f1_m: 0.8786 - val_loss: 0.6080 - val_accuracy: 0.8167 - val_f1_m: 0.8185
Epoch 86/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0208 - accuracy: 0.8814 - f1_m: 0.8783 - val_loss: 0.5813 - val_accuracy: 0.8327 - val_f1_m: 0.8338
Epoch 87/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0204 - accuracy: 0.8844 - f1_m: 0.8852 - val_loss: 0.5793 - val_accuracy: 0.8273 - val_f1_m: 0.8285
Epoch 88/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0212 - accuracy: 0.8788 - f1_m: 0.8797 - val_loss: 0.6321 - val_accuracy: 0.8143 - val_f1_m: 0.8150
Epoch 89/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0208 - accuracy: 0.8824 - f1_m: 0.8814 - val_loss: 0.5613 - val_accuracy: 0.8337 - val_f1_m: 0.8329
Epoch 90/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0189 - accuracy: 0.8906 - f1_m: 0.8903 - val_loss: 0.5963 - val_accuracy: 0.8183 - val_f1_m: 0.8229
Epoch 91/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0216 - accuracy: 0.8793 - f1_m: 0.8786 - val_loss: 0.5845 - val_accuracy: 0.8247 - val_f1_m: 0.8269
Epoch 92/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0190 - accuracy: 0.8909 - f1_m: 0.8915 - val_loss: 0.5597 - val_accuracy: 0.8327 - val_f1_m: 0.8356
Epoch 93/150
142/142 [==============================] - 2s 10ms/step - loss: 0.0186 - accuracy: 0.8963 - f1_m: 0.8949 - val_loss: 0.6021 - val_accuracy: 0.8160 - val_f1_m: 0.8205
Epoch 94/150
142/142 [==============================] - 2s 10ms/step - loss: 0.0187 - accuracy: 0.8931 - f1_m: 0.8897 - val_loss: 0.6277 - val_accuracy: 0.8107 - val_f1_m: 0.8152
Epoch 94: early stopping
CPU times: total: 2min 1s
Wall time: 2min 26s

evaluate_and_save_model(BaseModel_128_DA,  'BaseModel_128_DA', 64,save_models=False)
evaluate_and_save_model(BaseModel_31_DA, 'BaseModel_31_DA', 64,save_models=False)

Model Name            BaseModel_128_DA
Epochs                              55
Batch Size                          64
Train Loss                     0.00829
Val Loss                      0.602679
Train Acc                     0.957466
Val Acc                          0.842
[Train - Val] Acc             0.115466
[Train - Val] Loss           -0.594389
Name: BaseModel_128_DA, dtype: object
Model Name            BaseModel_31_DA
Epochs                             94
Batch Size                         64
Train Loss                   0.020766
Val Loss                     0.561303
Train Acc                    0.882366
Val Acc                      0.833667
[Train - Val] Acc            0.048699
[Train - Val] Loss          -0.540537
Name: BaseModel_31_DA, dtype: object

DAbaseline_128_history = json.load(open('./models/BaseModel_128_DA_history.json', 'r'))
DAbaseline_31_history = json.load(open('./models/BaseModel_31_DA_history.json', 'r'))

compare_curves(DAbaseline_31_history, DAbaseline_128_history, '31x31', '128x128')

Observations:

• We can see that comparing to the model without data agumentation, the model perfoms worse. However looking at the accuracy and loss graphs, we can see that the model does not overfit as badly as the model trained without data augmentation.
• We still see that the 128x128 model overfitts alot more and faster as compared to our 31x31 model

Training with Data Augmentation and Regularization

def create_Basemodel_regularization(input_shape, learning_rate, momentum, class_weights_dict, dataset_train, dataset_test, steps_per_epoch, weight_decay):
    tf.keras.backend.clear_session()
    model = Sequential()
    model.add(tf.keras.layers.Rescaling(1./255, input_shape=input_shape))
    model.add(Conv2D(64, (3,3), activation='relu', kernel_regularizer=l2(weight_decay)))
    model.add(MaxPooling2D((2,2)))
    model.add(Conv2D(64, (3,3), activation='relu', kernel_regularizer=l2(weight_decay)))
    model.add(MaxPooling2D((2,2)))
    model.add(Flatten())
    model.add(Dense(128, activation='relu', kernel_regularizer=l2(weight_decay)))
    model.add(Dense(128, activation='relu', kernel_regularizer=l2(weight_decay)))
    model.add(Dense(15, activation='softmax'))
    model.compile(optimizer=SGD(learning_rate=learning_rate, momentum=momentum), loss='categorical_crossentropy', metrics=['accuracy',f1_m])
    model.fit(dataset_train, epochs=150, validation_data=dataset_test, class_weight=class_weights_dict,callbacks=[EarlyStopping(monitor='val_loss', patience=10, verbose=1, mode='min')],steps_per_epoch=steps_per_epoch)
    return model

%%time
BaseModel_31_DA_RE = create_Basemodel_regularization((31,31,1),learning_rate,momentum,class_weights_dict,dataset_train_31,dataset_test_31,steps_per_epoch,weight_decay)

Epoch 1/150
142/142 [==============================] - 2s 12ms/step - loss: 0.3631 - accuracy: 0.0463 - f1_m: 0.0000e+00 - val_loss: 2.9083 - val_accuracy: 0.0780 - val_f1_m: 0.0000e+00
Epoch 2/150
142/142 [==============================] - 2s 11ms/step - loss: 0.3511 - accuracy: 0.0757 - f1_m: 0.0000e+00 - val_loss: 2.8917 - val_accuracy: 0.1077 - val_f1_m: 0.0000e+00
Epoch 3/150
142/142 [==============================] - 2s 11ms/step - loss: 0.3398 - accuracy: 0.0804 - f1_m: 0.0000e+00 - val_loss: 2.8754 - val_accuracy: 0.1243 - val_f1_m: 0.0000e+00
Epoch 4/150
142/142 [==============================] - 2s 12ms/step - loss: 0.3290 - accuracy: 0.1032 - f1_m: 0.0000e+00 - val_loss: 2.8571 - val_accuracy: 0.1327 - val_f1_m: 0.0000e+00
Epoch 5/150
142/142 [==============================] - 2s 11ms/step - loss: 0.3187 - accuracy: 0.1177 - f1_m: 0.0000e+00 - val_loss: 2.8357 - val_accuracy: 0.1410 - val_f1_m: 0.0000e+00
Epoch 6/150
142/142 [==============================] - 2s 11ms/step - loss: 0.3087 - accuracy: 0.1283 - f1_m: 0.0000e+00 - val_loss: 2.8080 - val_accuracy: 0.1430 - val_f1_m: 0.0000e+00
Epoch 7/150
142/142 [==============================] - 2s 10ms/step - loss: 0.2987 - accuracy: 0.1303 - f1_m: 0.0000e+00 - val_loss: 2.7681 - val_accuracy: 0.1467 - val_f1_m: 0.0000e+00
Epoch 8/150
142/142 [==============================] - 2s 10ms/step - loss: 0.2885 - accuracy: 0.1361 - f1_m: 0.0000e+00 - val_loss: 2.7059 - val_accuracy: 0.1697 - val_f1_m: 0.0000e+00
Epoch 9/150
142/142 [==============================] - 2s 11ms/step - loss: 0.2779 - accuracy: 0.1613 - f1_m: 0.0000e+00 - val_loss: 2.6281 - val_accuracy: 0.2170 - val_f1_m: 0.0000e+00
Epoch 10/150
142/142 [==============================] - 2s 11ms/step - loss: 0.2671 - accuracy: 0.1902 - f1_m: 0.0000e+00 - val_loss: 2.5486 - val_accuracy: 0.2327 - val_f1_m: 0.0000e+00
Epoch 11/150
142/142 [==============================] - 2s 11ms/step - loss: 0.2573 - accuracy: 0.2117 - f1_m: 0.0026 - val_loss: 2.4952 - val_accuracy: 0.2350 - val_f1_m: 0.0206
Epoch 12/150
142/142 [==============================] - 2s 10ms/step - loss: 0.2482 - accuracy: 0.2383 - f1_m: 0.0054 - val_loss: 2.4324 - val_accuracy: 0.2600 - val_f1_m: 0.0224
Epoch 13/150
142/142 [==============================] - 2s 10ms/step - loss: 0.2401 - accuracy: 0.2590 - f1_m: 0.0066 - val_loss: 2.3980 - val_accuracy: 0.2653 - val_f1_m: 0.0168
Epoch 14/150
142/142 [==============================] - 1s 10ms/step - loss: 0.2332 - accuracy: 0.2641 - f1_m: 0.0077 - val_loss: 2.3730 - val_accuracy: 0.2630 - val_f1_m: 0.0224
Epoch 15/150
142/142 [==============================] - 1s 10ms/step - loss: 0.2268 - accuracy: 0.2705 - f1_m: 0.0130 - val_loss: 2.3364 - val_accuracy: 0.2787 - val_f1_m: 0.0284
Epoch 16/150
142/142 [==============================] - 1s 10ms/step - loss: 0.2203 - accuracy: 0.2815 - f1_m: 0.0149 - val_loss: 2.3098 - val_accuracy: 0.2853 - val_f1_m: 0.0327
Epoch 17/150
142/142 [==============================] - 1s 10ms/step - loss: 0.2142 - accuracy: 0.2971 - f1_m: 0.0177 - val_loss: 2.2912 - val_accuracy: 0.2893 - val_f1_m: 0.0342
Epoch 18/150
142/142 [==============================] - 2s 10ms/step - loss: 0.2079 - accuracy: 0.3027 - f1_m: 0.0258 - val_loss: 2.2851 - val_accuracy: 0.2850 - val_f1_m: 0.0619
Epoch 19/150
142/142 [==============================] - 2s 11ms/step - loss: 0.2024 - accuracy: 0.3119 - f1_m: 0.0326 - val_loss: 2.2353 - val_accuracy: 0.3013 - val_f1_m: 0.0681
Epoch 20/150
142/142 [==============================] - 2s 11ms/step - loss: 0.1973 - accuracy: 0.3168 - f1_m: 0.0376 - val_loss: 2.2071 - val_accuracy: 0.3270 - val_f1_m: 0.0696
Epoch 21/150
142/142 [==============================] - 2s 11ms/step - loss: 0.1915 - accuracy: 0.3358 - f1_m: 0.0538 - val_loss: 2.1806 - val_accuracy: 0.3357 - val_f1_m: 0.0623
Epoch 22/150
142/142 [==============================] - 2s 11ms/step - loss: 0.1866 - accuracy: 0.3436 - f1_m: 0.0626 - val_loss: 2.1678 - val_accuracy: 0.3227 - val_f1_m: 0.1013
Epoch 23/150
142/142 [==============================] - 2s 11ms/step - loss: 0.1810 - accuracy: 0.3587 - f1_m: 0.0767 - val_loss: 2.0720 - val_accuracy: 0.3593 - val_f1_m: 0.0997
Epoch 24/150
142/142 [==============================] - 2s 11ms/step - loss: 0.1760 - accuracy: 0.3705 - f1_m: 0.0911 - val_loss: 2.0400 - val_accuracy: 0.3730 - val_f1_m: 0.1397
Epoch 25/150
142/142 [==============================] - 2s 11ms/step - loss: 0.1718 - accuracy: 0.3762 - f1_m: 0.1168 - val_loss: 1.9876 - val_accuracy: 0.3833 - val_f1_m: 0.1500
Epoch 26/150
142/142 [==============================] - 2s 11ms/step - loss: 0.1667 - accuracy: 0.3879 - f1_m: 0.1382 - val_loss: 1.9518 - val_accuracy: 0.3987 - val_f1_m: 0.1742
Epoch 27/150
142/142 [==============================] - 2s 10ms/step - loss: 0.1623 - accuracy: 0.4008 - f1_m: 0.1639 - val_loss: 2.0443 - val_accuracy: 0.3677 - val_f1_m: 0.2070
Epoch 28/150
142/142 [==============================] - 1s 10ms/step - loss: 0.1595 - accuracy: 0.4005 - f1_m: 0.1713 - val_loss: 1.9390 - val_accuracy: 0.3930 - val_f1_m: 0.2318
Epoch 29/150
142/142 [==============================] - 1s 10ms/step - loss: 0.1545 - accuracy: 0.4147 - f1_m: 0.1989 - val_loss: 1.8599 - val_accuracy: 0.4193 - val_f1_m: 0.2444
Epoch 30/150
142/142 [==============================] - 1s 10ms/step - loss: 0.1498 - accuracy: 0.4318 - f1_m: 0.2260 - val_loss: 1.8589 - val_accuracy: 0.4167 - val_f1_m: 0.2263
Epoch 31/150
142/142 [==============================] - 1s 10ms/step - loss: 0.1478 - accuracy: 0.4348 - f1_m: 0.2306 - val_loss: 1.7975 - val_accuracy: 0.4400 - val_f1_m: 0.2547
Epoch 32/150
142/142 [==============================] - 2s 10ms/step - loss: 0.1433 - accuracy: 0.4516 - f1_m: 0.2588 - val_loss: 1.8770 - val_accuracy: 0.4137 - val_f1_m: 0.2437
Epoch 33/150
142/142 [==============================] - 1s 10ms/step - loss: 0.1406 - accuracy: 0.4578 - f1_m: 0.2786 - val_loss: 1.7287 - val_accuracy: 0.4690 - val_f1_m: 0.3046
Epoch 34/150
142/142 [==============================] - 1s 10ms/step - loss: 0.1370 - accuracy: 0.4724 - f1_m: 0.2961 - val_loss: 1.7169 - val_accuracy: 0.4763 - val_f1_m: 0.3192
Epoch 35/150
142/142 [==============================] - 1s 10ms/step - loss: 0.1343 - accuracy: 0.4733 - f1_m: 0.3105 - val_loss: 1.6893 - val_accuracy: 0.4807 - val_f1_m: 0.3389
Epoch 36/150
142/142 [==============================] - 1s 10ms/step - loss: 0.1313 - accuracy: 0.4907 - f1_m: 0.3353 - val_loss: 1.6938 - val_accuracy: 0.4773 - val_f1_m: 0.3333
Epoch 37/150
142/142 [==============================] - 1s 10ms/step - loss: 0.1292 - accuracy: 0.4993 - f1_m: 0.3398 - val_loss: 1.7823 - val_accuracy: 0.4323 - val_f1_m: 0.3209
Epoch 38/150
142/142 [==============================] - 2s 10ms/step - loss: 0.1264 - accuracy: 0.5071 - f1_m: 0.3607 - val_loss: 1.6024 - val_accuracy: 0.5040 - val_f1_m: 0.3869
Epoch 39/150
142/142 [==============================] - 2s 10ms/step - loss: 0.1238 - accuracy: 0.5132 - f1_m: 0.3756 - val_loss: 1.5919 - val_accuracy: 0.5083 - val_f1_m: 0.3960
Epoch 40/150
142/142 [==============================] - 2s 11ms/step - loss: 0.1213 - accuracy: 0.5247 - f1_m: 0.3954 - val_loss: 1.5924 - val_accuracy: 0.5150 - val_f1_m: 0.3898
Epoch 41/150
142/142 [==============================] - 2s 11ms/step - loss: 0.1189 - accuracy: 0.5337 - f1_m: 0.4116 - val_loss: 1.5287 - val_accuracy: 0.5237 - val_f1_m: 0.4156
Epoch 42/150
142/142 [==============================] - 2s 11ms/step - loss: 0.1169 - accuracy: 0.5436 - f1_m: 0.4216 - val_loss: 1.5916 - val_accuracy: 0.4950 - val_f1_m: 0.4051
Epoch 43/150
142/142 [==============================] - 2s 11ms/step - loss: 0.1145 - accuracy: 0.5480 - f1_m: 0.4462 - val_loss: 1.4714 - val_accuracy: 0.5510 - val_f1_m: 0.4538
Epoch 44/150
142/142 [==============================] - 2s 11ms/step - loss: 0.1126 - accuracy: 0.5554 - f1_m: 0.4515 - val_loss: 1.5734 - val_accuracy: 0.5070 - val_f1_m: 0.4305
Epoch 45/150
142/142 [==============================] - 2s 11ms/step - loss: 0.1102 - accuracy: 0.5691 - f1_m: 0.4733 - val_loss: 1.5312 - val_accuracy: 0.5197 - val_f1_m: 0.4514
Epoch 46/150
142/142 [==============================] - 2s 10ms/step - loss: 0.1088 - accuracy: 0.5714 - f1_m: 0.4812 - val_loss: 1.4789 - val_accuracy: 0.5487 - val_f1_m: 0.4600
Epoch 47/150
142/142 [==============================] - 2s 10ms/step - loss: 0.1078 - accuracy: 0.5739 - f1_m: 0.4804 - val_loss: 1.4763 - val_accuracy: 0.5403 - val_f1_m: 0.4782
Epoch 48/150
142/142 [==============================] - 2s 10ms/step - loss: 0.1060 - accuracy: 0.5836 - f1_m: 0.5020 - val_loss: 1.4026 - val_accuracy: 0.5653 - val_f1_m: 0.4994
Epoch 49/150
142/142 [==============================] - 2s 11ms/step - loss: 0.1031 - accuracy: 0.6057 - f1_m: 0.5149 - val_loss: 1.3638 - val_accuracy: 0.5930 - val_f1_m: 0.5194
Epoch 50/150
142/142 [==============================] - 2s 10ms/step - loss: 0.1025 - accuracy: 0.6043 - f1_m: 0.5231 - val_loss: 1.3934 - val_accuracy: 0.5793 - val_f1_m: 0.5009
Epoch 51/150
142/142 [==============================] - 1s 10ms/step - loss: 0.1002 - accuracy: 0.6180 - f1_m: 0.5410 - val_loss: 1.3724 - val_accuracy: 0.5777 - val_f1_m: 0.5206
Epoch 52/150
142/142 [==============================] - 2s 11ms/step - loss: 0.0993 - accuracy: 0.6148 - f1_m: 0.5492 - val_loss: 1.3464 - val_accuracy: 0.6023 - val_f1_m: 0.5283
Epoch 53/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0967 - accuracy: 0.6342 - f1_m: 0.5616 - val_loss: 1.3317 - val_accuracy: 0.5977 - val_f1_m: 0.5360
Epoch 54/150
142/142 [==============================] - 2s 11ms/step - loss: 0.0963 - accuracy: 0.6365 - f1_m: 0.5650 - val_loss: 1.2894 - val_accuracy: 0.6160 - val_f1_m: 0.5610
Epoch 55/150
142/142 [==============================] - 2s 11ms/step - loss: 0.0952 - accuracy: 0.6388 - f1_m: 0.5726 - val_loss: 1.3018 - val_accuracy: 0.6227 - val_f1_m: 0.5632
Epoch 56/150
142/142 [==============================] - 2s 10ms/step - loss: 0.0931 - accuracy: 0.6522 - f1_m: 0.5863 - val_loss: 1.2745 - val_accuracy: 0.6117 - val_f1_m: 0.5692
Epoch 57/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0917 - accuracy: 0.6596 - f1_m: 0.5968 - val_loss: 1.3920 - val_accuracy: 0.5713 - val_f1_m: 0.5252
Epoch 58/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0929 - accuracy: 0.6470 - f1_m: 0.5873 - val_loss: 1.2412 - val_accuracy: 0.6373 - val_f1_m: 0.5841
Epoch 59/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0898 - accuracy: 0.6678 - f1_m: 0.6135 - val_loss: 1.2483 - val_accuracy: 0.6330 - val_f1_m: 0.5793
Epoch 60/150
142/142 [==============================] - 2s 11ms/step - loss: 0.0882 - accuracy: 0.6742 - f1_m: 0.6200 - val_loss: 1.2591 - val_accuracy: 0.6240 - val_f1_m: 0.5771
Epoch 61/150
142/142 [==============================] - 2s 11ms/step - loss: 0.0886 - accuracy: 0.6707 - f1_m: 0.6254 - val_loss: 1.1887 - val_accuracy: 0.6603 - val_f1_m: 0.6070
Epoch 62/150
142/142 [==============================] - 2s 11ms/step - loss: 0.0864 - accuracy: 0.6906 - f1_m: 0.6368 - val_loss: 1.3977 - val_accuracy: 0.5783 - val_f1_m: 0.5399
Epoch 63/150
142/142 [==============================] - 2s 11ms/step - loss: 0.0898 - accuracy: 0.6717 - f1_m: 0.6160 - val_loss: 1.1795 - val_accuracy: 0.6580 - val_f1_m: 0.6181
Epoch 64/150
142/142 [==============================] - 2s 11ms/step - loss: 0.0840 - accuracy: 0.7040 - f1_m: 0.6539 - val_loss: 1.2333 - val_accuracy: 0.6257 - val_f1_m: 0.5797
Epoch 65/150
142/142 [==============================] - 2s 11ms/step - loss: 0.0836 - accuracy: 0.7029 - f1_m: 0.6610 - val_loss: 1.1573 - val_accuracy: 0.6627 - val_f1_m: 0.6321
Epoch 66/150
142/142 [==============================] - 2s 11ms/step - loss: 0.0834 - accuracy: 0.7065 - f1_m: 0.6526 - val_loss: 1.2101 - val_accuracy: 0.6350 - val_f1_m: 0.5913
Epoch 67/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0830 - accuracy: 0.7065 - f1_m: 0.6658 - val_loss: 1.1503 - val_accuracy: 0.6600 - val_f1_m: 0.6347
Epoch 68/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0813 - accuracy: 0.7149 - f1_m: 0.6797 - val_loss: 1.1894 - val_accuracy: 0.6530 - val_f1_m: 0.6201
Epoch 69/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0805 - accuracy: 0.7200 - f1_m: 0.6885 - val_loss: 1.1504 - val_accuracy: 0.6687 - val_f1_m: 0.6345
Epoch 70/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0788 - accuracy: 0.7360 - f1_m: 0.6957 - val_loss: 1.1375 - val_accuracy: 0.6603 - val_f1_m: 0.6365
Epoch 71/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0780 - accuracy: 0.7395 - f1_m: 0.7069 - val_loss: 1.0894 - val_accuracy: 0.6800 - val_f1_m: 0.6583
Epoch 72/150
142/142 [==============================] - 2s 11ms/step - loss: 0.0761 - accuracy: 0.7506 - f1_m: 0.7196 - val_loss: 1.1163 - val_accuracy: 0.6743 - val_f1_m: 0.6570
Epoch 73/150
142/142 [==============================] - 2s 11ms/step - loss: 0.0760 - accuracy: 0.7498 - f1_m: 0.7213 - val_loss: 1.0657 - val_accuracy: 0.6920 - val_f1_m: 0.6751
Epoch 74/150
142/142 [==============================] - 2s 12ms/step - loss: 0.0753 - accuracy: 0.7586 - f1_m: 0.7306 - val_loss: 1.0490 - val_accuracy: 0.6930 - val_f1_m: 0.6744
Epoch 75/150
142/142 [==============================] - 2s 11ms/step - loss: 0.0741 - accuracy: 0.7686 - f1_m: 0.7332 - val_loss: 1.0088 - val_accuracy: 0.7083 - val_f1_m: 0.6943
Epoch 76/150
142/142 [==============================] - 2s 11ms/step - loss: 0.0740 - accuracy: 0.7669 - f1_m: 0.7399 - val_loss: 1.0436 - val_accuracy: 0.7023 - val_f1_m: 0.6753
Epoch 77/150
142/142 [==============================] - 2s 11ms/step - loss: 0.0730 - accuracy: 0.7723 - f1_m: 0.7461 - val_loss: 1.1590 - val_accuracy: 0.6533 - val_f1_m: 0.6428
Epoch 78/150
142/142 [==============================] - 2s 11ms/step - loss: 0.0731 - accuracy: 0.7749 - f1_m: 0.7475 - val_loss: 1.0821 - val_accuracy: 0.6813 - val_f1_m: 0.6662
Epoch 79/150
142/142 [==============================] - 2s 12ms/step - loss: 0.0714 - accuracy: 0.7838 - f1_m: 0.7597 - val_loss: 1.0907 - val_accuracy: 0.6713 - val_f1_m: 0.6523
Epoch 80/150
142/142 [==============================] - 2s 12ms/step - loss: 0.0719 - accuracy: 0.7788 - f1_m: 0.7603 - val_loss: 0.9532 - val_accuracy: 0.7270 - val_f1_m: 0.7174
Epoch 81/150
142/142 [==============================] - 2s 12ms/step - loss: 0.0702 - accuracy: 0.7889 - f1_m: 0.7696 - val_loss: 0.9981 - val_accuracy: 0.7073 - val_f1_m: 0.7034
Epoch 82/150
142/142 [==============================] - 2s 11ms/step - loss: 0.0702 - accuracy: 0.7930 - f1_m: 0.7736 - val_loss: 1.0340 - val_accuracy: 0.6987 - val_f1_m: 0.6847
Epoch 83/150
142/142 [==============================] - 2s 11ms/step - loss: 0.0689 - accuracy: 0.8004 - f1_m: 0.7788 - val_loss: 0.9359 - val_accuracy: 0.7287 - val_f1_m: 0.7183
Epoch 84/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0690 - accuracy: 0.7975 - f1_m: 0.7761 - val_loss: 0.9874 - val_accuracy: 0.7190 - val_f1_m: 0.7085
Epoch 85/150
142/142 [==============================] - 2s 11ms/step - loss: 0.0690 - accuracy: 0.8000 - f1_m: 0.7755 - val_loss: 0.9288 - val_accuracy: 0.7350 - val_f1_m: 0.7313
Epoch 86/150
142/142 [==============================] - 2s 10ms/step - loss: 0.0687 - accuracy: 0.8016 - f1_m: 0.7801 - val_loss: 0.9395 - val_accuracy: 0.7287 - val_f1_m: 0.7223
Epoch 87/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0674 - accuracy: 0.8113 - f1_m: 0.7916 - val_loss: 0.9339 - val_accuracy: 0.7387 - val_f1_m: 0.7196
Epoch 88/150
142/142 [==============================] - 2s 12ms/step - loss: 0.0666 - accuracy: 0.8196 - f1_m: 0.7995 - val_loss: 0.9580 - val_accuracy: 0.7273 - val_f1_m: 0.7161
Epoch 89/150
142/142 [==============================] - 2s 11ms/step - loss: 0.0660 - accuracy: 0.8200 - f1_m: 0.8004 - val_loss: 0.9419 - val_accuracy: 0.7297 - val_f1_m: 0.7233
Epoch 90/150
142/142 [==============================] - 2s 11ms/step - loss: 0.0666 - accuracy: 0.8207 - f1_m: 0.8004 - val_loss: 1.1195 - val_accuracy: 0.6727 - val_f1_m: 0.6497
Epoch 91/150
142/142 [==============================] - 2s 12ms/step - loss: 0.0676 - accuracy: 0.8115 - f1_m: 0.7989 - val_loss: 0.9161 - val_accuracy: 0.7433 - val_f1_m: 0.7373
Epoch 92/150
142/142 [==============================] - 2s 12ms/step - loss: 0.0645 - accuracy: 0.8295 - f1_m: 0.8163 - val_loss: 0.8970 - val_accuracy: 0.7330 - val_f1_m: 0.7324
Epoch 93/150
142/142 [==============================] - 2s 11ms/step - loss: 0.0640 - accuracy: 0.8385 - f1_m: 0.8204 - val_loss: 0.8959 - val_accuracy: 0.7510 - val_f1_m: 0.7429
Epoch 94/150
142/142 [==============================] - 2s 10ms/step - loss: 0.0636 - accuracy: 0.8398 - f1_m: 0.8244 - val_loss: 1.4933 - val_accuracy: 0.5550 - val_f1_m: 0.5495
Epoch 95/150
142/142 [==============================] - 2s 11ms/step - loss: 0.0676 - accuracy: 0.8119 - f1_m: 0.7941 - val_loss: 0.8672 - val_accuracy: 0.7533 - val_f1_m: 0.7456
Epoch 96/150
142/142 [==============================] - 2s 10ms/step - loss: 0.0625 - accuracy: 0.8517 - f1_m: 0.8384 - val_loss: 0.9030 - val_accuracy: 0.7413 - val_f1_m: 0.7337
Epoch 97/150
142/142 [==============================] - 2s 10ms/step - loss: 0.0620 - accuracy: 0.8525 - f1_m: 0.8403 - val_loss: 0.9809 - val_accuracy: 0.7127 - val_f1_m: 0.7087
Epoch 98/150
142/142 [==============================] - 2s 10ms/step - loss: 0.0626 - accuracy: 0.8489 - f1_m: 0.8368 - val_loss: 0.8436 - val_accuracy: 0.7597 - val_f1_m: 0.7568
Epoch 99/150
142/142 [==============================] - 2s 10ms/step - loss: 0.0608 - accuracy: 0.8603 - f1_m: 0.8472 - val_loss: 0.8854 - val_accuracy: 0.7603 - val_f1_m: 0.7487
Epoch 100/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0624 - accuracy: 0.8480 - f1_m: 0.8350 - val_loss: 0.8894 - val_accuracy: 0.7463 - val_f1_m: 0.7420
Epoch 101/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0604 - accuracy: 0.8618 - f1_m: 0.8499 - val_loss: 1.2201 - val_accuracy: 0.6547 - val_f1_m: 0.6414
Epoch 102/150
142/142 [==============================] - 2s 10ms/step - loss: 0.0638 - accuracy: 0.8419 - f1_m: 0.8255 - val_loss: 0.8682 - val_accuracy: 0.7567 - val_f1_m: 0.7500
Epoch 103/150
142/142 [==============================] - 2s 10ms/step - loss: 0.0600 - accuracy: 0.8694 - f1_m: 0.8594 - val_loss: 0.8565 - val_accuracy: 0.7617 - val_f1_m: 0.7553
Epoch 104/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0599 - accuracy: 0.8689 - f1_m: 0.8535 - val_loss: 0.9009 - val_accuracy: 0.7483 - val_f1_m: 0.7424
Epoch 105/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0599 - accuracy: 0.8652 - f1_m: 0.8571 - val_loss: 0.8841 - val_accuracy: 0.7440 - val_f1_m: 0.7414
Epoch 106/150
142/142 [==============================] - 2s 10ms/step - loss: 0.0593 - accuracy: 0.8732 - f1_m: 0.8610 - val_loss: 0.8497 - val_accuracy: 0.7583 - val_f1_m: 0.7622
Epoch 107/150
142/142 [==============================] - 2s 10ms/step - loss: 0.0594 - accuracy: 0.8725 - f1_m: 0.8651 - val_loss: 0.8737 - val_accuracy: 0.7580 - val_f1_m: 0.7508
Epoch 108/150
142/142 [==============================] - 1s 10ms/step - loss: 0.0588 - accuracy: 0.8767 - f1_m: 0.8635 - val_loss: 0.8585 - val_accuracy: 0.7610 - val_f1_m: 0.7565
Epoch 108: early stopping
CPU times: total: 2min 23s
Wall time: 2min 51s

%%time
BaseModel_128_DA_RE = create_Basemodel_regularization((128,128,1),learning_rate,momentum,class_weights_dict,dataset_train_128,dataset_test_128,steps_per_epoch,weight_decay)

Epoch 1/150
142/142 [==============================] - 6s 37ms/step - loss: 0.3695 - accuracy: 0.0944 - f1_m: 0.0000e+00 - val_loss: 2.8922 - val_accuracy: 0.1210 - val_f1_m: 0.0000e+00
Epoch 2/150
142/142 [==============================] - 5s 36ms/step - loss: 0.3544 - accuracy: 0.1306 - f1_m: 0.0000e+00 - val_loss: 2.7994 - val_accuracy: 0.1320 - val_f1_m: 0.0000e+00
Epoch 3/150
142/142 [==============================] - 5s 36ms/step - loss: 0.3374 - accuracy: 0.1628 - f1_m: 0.0035 - val_loss: 2.7085 - val_accuracy: 0.1573 - val_f1_m: 0.0234
Epoch 4/150
142/142 [==============================] - 5s 35ms/step - loss: 0.3211 - accuracy: 0.2255 - f1_m: 0.0075 - val_loss: 2.5500 - val_accuracy: 0.2683 - val_f1_m: 0.0198
Epoch 5/150
142/142 [==============================] - 5s 36ms/step - loss: 0.3032 - accuracy: 0.2878 - f1_m: 0.0176 - val_loss: 2.3888 - val_accuracy: 0.2990 - val_f1_m: 0.0578
Epoch 6/150
142/142 [==============================] - 5s 36ms/step - loss: 0.2867 - accuracy: 0.3373 - f1_m: 0.0576 - val_loss: 2.3293 - val_accuracy: 0.3120 - val_f1_m: 0.1389
Epoch 7/150
142/142 [==============================] - 5s 36ms/step - loss: 0.2713 - accuracy: 0.3724 - f1_m: 0.1286 - val_loss: 2.1661 - val_accuracy: 0.3760 - val_f1_m: 0.1781
Epoch 8/150
142/142 [==============================] - 5s 36ms/step - loss: 0.2567 - accuracy: 0.4086 - f1_m: 0.1972 - val_loss: 2.0725 - val_accuracy: 0.3960 - val_f1_m: 0.2583
Epoch 9/150
142/142 [==============================] - 5s 36ms/step - loss: 0.2429 - accuracy: 0.4516 - f1_m: 0.2565 - val_loss: 1.9500 - val_accuracy: 0.4370 - val_f1_m: 0.2982
Epoch 10/150
142/142 [==============================] - 5s 36ms/step - loss: 0.2291 - accuracy: 0.4867 - f1_m: 0.3316 - val_loss: 1.7864 - val_accuracy: 0.4903 - val_f1_m: 0.3737
Epoch 11/150
142/142 [==============================] - 5s 36ms/step - loss: 0.2152 - accuracy: 0.5253 - f1_m: 0.4104 - val_loss: 1.7593 - val_accuracy: 0.4920 - val_f1_m: 0.4112
Epoch 12/150
142/142 [==============================] - 5s 36ms/step - loss: 0.2015 - accuracy: 0.5677 - f1_m: 0.4828 - val_loss: 1.5988 - val_accuracy: 0.5387 - val_f1_m: 0.4747
Epoch 13/150
142/142 [==============================] - 5s 36ms/step - loss: 0.1897 - accuracy: 0.6061 - f1_m: 0.5302 - val_loss: 1.3972 - val_accuracy: 0.5957 - val_f1_m: 0.5484
Epoch 14/150
142/142 [==============================] - 5s 36ms/step - loss: 0.1796 - accuracy: 0.6321 - f1_m: 0.5791 - val_loss: 1.3273 - val_accuracy: 0.6260 - val_f1_m: 0.5818
Epoch 15/150
142/142 [==============================] - 5s 36ms/step - loss: 0.1699 - accuracy: 0.6627 - f1_m: 0.6210 - val_loss: 1.5478 - val_accuracy: 0.5570 - val_f1_m: 0.5203
Epoch 16/150
142/142 [==============================] - 5s 36ms/step - loss: 0.1580 - accuracy: 0.7029 - f1_m: 0.6690 - val_loss: 1.3066 - val_accuracy: 0.6140 - val_f1_m: 0.6110
Epoch 17/150
142/142 [==============================] - 5s 36ms/step - loss: 0.1489 - accuracy: 0.7280 - f1_m: 0.7077 - val_loss: 1.1321 - val_accuracy: 0.6883 - val_f1_m: 0.6704
Epoch 18/150
142/142 [==============================] - 5s 36ms/step - loss: 0.1394 - accuracy: 0.7622 - f1_m: 0.7404 - val_loss: 1.2173 - val_accuracy: 0.6617 - val_f1_m: 0.6509
Epoch 19/150
142/142 [==============================] - 5s 36ms/step - loss: 0.1346 - accuracy: 0.7644 - f1_m: 0.7460 - val_loss: 1.1868 - val_accuracy: 0.6637 - val_f1_m: 0.6603
Epoch 20/150
142/142 [==============================] - 5s 36ms/step - loss: 0.1268 - accuracy: 0.7950 - f1_m: 0.7813 - val_loss: 1.3124 - val_accuracy: 0.6393 - val_f1_m: 0.6312
Epoch 21/150
142/142 [==============================] - 5s 36ms/step - loss: 0.1201 - accuracy: 0.8111 - f1_m: 0.8005 - val_loss: 1.1900 - val_accuracy: 0.6690 - val_f1_m: 0.6634
Epoch 22/150
142/142 [==============================] - 5s 36ms/step - loss: 0.1160 - accuracy: 0.8178 - f1_m: 0.8117 - val_loss: 1.0923 - val_accuracy: 0.6967 - val_f1_m: 0.6993
Epoch 23/150
142/142 [==============================] - 5s 36ms/step - loss: 0.1081 - accuracy: 0.8481 - f1_m: 0.8402 - val_loss: 1.0908 - val_accuracy: 0.7030 - val_f1_m: 0.7020
Epoch 24/150
142/142 [==============================] - 5s 36ms/step - loss: 0.1032 - accuracy: 0.8658 - f1_m: 0.8537 - val_loss: 1.0913 - val_accuracy: 0.6980 - val_f1_m: 0.7040
Epoch 25/150
142/142 [==============================] - 5s 36ms/step - loss: 0.1037 - accuracy: 0.8466 - f1_m: 0.8423 - val_loss: 1.0769 - val_accuracy: 0.7083 - val_f1_m: 0.7151
Epoch 26/150
142/142 [==============================] - 5s 36ms/step - loss: 0.0934 - accuracy: 0.8917 - f1_m: 0.8883 - val_loss: 0.9446 - val_accuracy: 0.7447 - val_f1_m: 0.7541
Epoch 27/150
142/142 [==============================] - 5s 36ms/step - loss: 0.0885 - accuracy: 0.9066 - f1_m: 0.9007 - val_loss: 1.1106 - val_accuracy: 0.7040 - val_f1_m: 0.7017
Epoch 28/150
142/142 [==============================] - 5s 36ms/step - loss: 0.0878 - accuracy: 0.8932 - f1_m: 0.8898 - val_loss: 1.0574 - val_accuracy: 0.7183 - val_f1_m: 0.7220
Epoch 29/150
142/142 [==============================] - 5s 36ms/step - loss: 0.0809 - accuracy: 0.9257 - f1_m: 0.9215 - val_loss: 0.9309 - val_accuracy: 0.7523 - val_f1_m: 0.7651
Epoch 30/150
142/142 [==============================] - 5s 36ms/step - loss: 0.0780 - accuracy: 0.9289 - f1_m: 0.9257 - val_loss: 0.9241 - val_accuracy: 0.7663 - val_f1_m: 0.7698
Epoch 31/150
142/142 [==============================] - 5s 36ms/step - loss: 0.0734 - accuracy: 0.9444 - f1_m: 0.9408 - val_loss: 1.1026 - val_accuracy: 0.7163 - val_f1_m: 0.7236
Epoch 32/150
142/142 [==============================] - 5s 36ms/step - loss: 0.0727 - accuracy: 0.9385 - f1_m: 0.9363 - val_loss: 0.8984 - val_accuracy: 0.7760 - val_f1_m: 0.7771
Epoch 33/150
142/142 [==============================] - 5s 36ms/step - loss: 0.0682 - accuracy: 0.9548 - f1_m: 0.9501 - val_loss: 0.8885 - val_accuracy: 0.7723 - val_f1_m: 0.7791
Epoch 34/150
142/142 [==============================] - 5s 36ms/step - loss: 0.0665 - accuracy: 0.9539 - f1_m: 0.9488 - val_loss: 1.0299 - val_accuracy: 0.7443 - val_f1_m: 0.7476
Epoch 35/150
142/142 [==============================] - 5s 36ms/step - loss: 0.0649 - accuracy: 0.9531 - f1_m: 0.9490 - val_loss: 0.8919 - val_accuracy: 0.7773 - val_f1_m: 0.7837
Epoch 36/150
142/142 [==============================] - 5s 36ms/step - loss: 0.0607 - accuracy: 0.9715 - f1_m: 0.9669 - val_loss: 0.9326 - val_accuracy: 0.7610 - val_f1_m: 0.7675
Epoch 37/150
142/142 [==============================] - 5s 36ms/step - loss: 0.0581 - accuracy: 0.9767 - f1_m: 0.9744 - val_loss: 0.9773 - val_accuracy: 0.7547 - val_f1_m: 0.7576
Epoch 38/150
142/142 [==============================] - 5s 36ms/step - loss: 0.0574 - accuracy: 0.9661 - f1_m: 0.9664 - val_loss: 0.9749 - val_accuracy: 0.7503 - val_f1_m: 0.7514
Epoch 39/150
142/142 [==============================] - 5s 36ms/step - loss: 0.0583 - accuracy: 0.9524 - f1_m: 0.9492 - val_loss: 0.9934 - val_accuracy: 0.7437 - val_f1_m: 0.7495
Epoch 40/150
142/142 [==============================] - 5s 36ms/step - loss: 0.0544 - accuracy: 0.9723 - f1_m: 0.9710 - val_loss: 0.9045 - val_accuracy: 0.7730 - val_f1_m: 0.7781
Epoch 41/150
142/142 [==============================] - 5s 36ms/step - loss: 0.0522 - accuracy: 0.9793 - f1_m: 0.9767 - val_loss: 0.9081 - val_accuracy: 0.7773 - val_f1_m: 0.7797
Epoch 42/150
142/142 [==============================] - 5s 36ms/step - loss: 0.0504 - accuracy: 0.9802 - f1_m: 0.9771 - val_loss: 0.9759 - val_accuracy: 0.7557 - val_f1_m: 0.7639
Epoch 43/150
142/142 [==============================] - 5s 36ms/step - loss: 0.0485 - accuracy: 0.9846 - f1_m: 0.9827 - val_loss: 1.0259 - val_accuracy: 0.7433 - val_f1_m: 0.7465
Epoch 43: early stopping
CPU times: total: 2min 5s
Wall time: 3min 42s

evaluate_and_save_model(BaseModel_128_DA_RE, 'BaseModel_128_DA_RE', 64,save_models=False)
evaluate_and_save_model(BaseModel_31_DA_RE, 'BaseModel_31_DA_RE', 64,save_models=False)

Model Name            BaseModel_128_DA_RE
Epochs                                 43
Batch Size                             64
Train Loss                       0.064911
Val Loss                         0.891881
Train Acc                        0.953146
Val Acc                          0.777333
[Train - Val] Acc                0.175812
[Train - Val] Loss              -0.826971
Name: BaseModel_128_DA_RE, dtype: object
Model Name            BaseModel_31_DA_RE
Epochs                               108
Batch Size                            64
Train Loss                      0.059987
Val Loss                        0.856455
Train Acc                       0.869406
Val Acc                         0.761667
[Train - Val] Acc                0.10774
[Train - Val] Loss             -0.796468
Name: BaseModel_31_DA_RE, dtype: object

DAREbaseline_128_history = json.load(open('./models/BaseModel_128_DA_RE_history.json', 'r'))
DAREbaseline_31_history = json.load(open('./models/BaseModel_31_DA_RE_history.json', 'r'))

compare_curves(DAREbaseline_31_history, DAREbaseline_128_history, '31x31', '128x128')

Observations:

• We can see that with the inclusion of L2 Regularization, our models tend to not overfit as much as the previous models.
• We can see that our 31x31 model has improved alot compared with training on only augmented images. However, our 128x128 model does not improve at all and even deproves compared with training on only augmented images.

How can we suit our models to the different sizes?

As mentioned early, bigger input sizes tend to overfit more and have longer training times due to the increased amount of trainable parameters. We also need deeper networks to allow our 128x128 model to better extract complex features.

• We will be using the same stratagies to prevent overfitting in both 31x31 and 128x128 models
• We will introduce more layers for our 128x128 model as compared to our 31x31 model.
• We cant really do anything about the longer training time, unless we reduce our parameters compare to our 31x31 model, however that would defeat the purpose of making our 128x128 model deeper in the first place. :(

Custom CNN Network

We will be making a custom CNN model based on VGG-16 neural network while tweaking a few parameters. As our baseline model does not seem up to standard with its high overfitting rates even with early stopping.

Introduction to VGG-16:

VGG-16 is a convolutional neural network architecture proposed by the Visual Graphics Group (VGG) at the University of Oxford. It was introduced in the paper "Very Deep Convolutional Networks for Large-Scale Image Recognition" by Karen Simonyan and Andrew Zisserman in 2014 [3]. The VGG-16 architecture is known for its simplicity and uniform structure, consisting of 16 weight layers, including 13 convolutional layers and 3 fuly connected layers.

Why did i choose VGG-16?

VGG-16 gained popularity for its straightforward and regular design, featuring small 3x3 convolutional filters stacked on top of each other. This uniform architecture makes it easy to implement. Despite its simplicity, VGG-16 demonstrated impressive performance on various image recognition tasks.

Architecture of a regular VGG-16 network:

Source

In the VGG-16 architecture, the network is organized into five blocks, each containing a certain number of convolutional layers followed by max-pooling layers. The fully connected layers at the end are responsible for making predictions. The use of small convolutional filters allows the network to learn complex features, and the pooling layers help reduce spatial dimensions.

Difference between VGG-16 and ResNet:

While VGG-16 and ResNet are both popular architectures, they differ in their approach to handling information flow. VGG-16 relies on stacking convolutional layers, while ResNet introduces skip connections to address the vanishing/exploding gradient problem. In ResNet, the skip connections allow the gradient to flow directly through the network, mitigating issues associated with deep architectures.

How are we going to build our custom model?

Here are some changes i made to our custom model

Reducing the amount of filters:

• Decreases our computational load due to hardware limitations.
• We will be adjusting the number of filters in convolutional layers, reducing the overall number of trainable parameters.

Batch Normalization:

• Faster convergence and improved generalization.
• We will be applying Batch normalizing to every ConvoBlock.

Our Network Depth for different image size:

• We need to address overfitting for different input sizes (31x31 and 128x128) espically 128x128. As larger input sizes are more prone to overfitting.
• Implementation:
  • For 31x31 input size: Implement a 9-layered VG16 variant.
  • For 128x128 input size: Use a 12-layered VG16 variant to handle larger input dimensions more effectively.

Including dropout layer:

• Dropout randomly deactivates a portion of neurons during training, promoting the development of a more robust and generalized model. :)
• We will be integrating dropout layers before the output layer.

L2 regularization:

• Mitigate overfitting by penalizing large weights.
• We will be applying L2 regularization to both Conv2D layers and Fully Connected (FC) layers within the network.

The traditional concern is that the noise introduced by dropout during training may interfere with the normalization process of batch normalization. Batch normalization smoothens the optimization landscape and accelerates training, while dropout adds regularization by preventing overfitting.

def ConvoBlock(X, num_convs, num_filters):

    for _ in range(num_convs):
        X = Conv2D(num_filters, (3, 3), padding='same')(X)
        X = BatchNormalization()(X)
        X = Activation('relu')(X)

    X = MaxPooling2D(pool_size=(2, 2), strides=(2, 2))(X)

    return X

def create_Custom_model(input_shape, num_classes):
    tf.keras.backend.clear_session()
    if input_shape == (31,31,1):
        num_convs = [3,3,3]
        num_filters = [64,128,256]
    elif input_shape == (128,128,1):
        num_convs = [2,2,2,3,3]
        num_filters = [16,32,64,128,256]
    X_input = Input(input_shape)

    for i in range(len(num_convs)):
        if i == 0:
            X = ConvoBlock(X_input, num_convs[i], num_filters[i])
        else:
            X = ConvoBlock(X, num_convs[i], num_filters[i])

    X = Flatten()(X)

    # Add a dense layer for classification
    X = Dense(256, activation='relu',kernel_regularizer=l2(weight_decay))(X)
    X = Dense(256, activation='relu',kernel_regularizer=l2(weight_decay))(X)
    X = Dropout(0.1)(X)
    X = Dense(num_classes, activation='softmax')(X)

    # Create the model
    model = Model(inputs=X_input, outputs=X)

    return model

CustomModel_128 = create_Custom_model((128, 128, 1), 15)
CustomModel_31 = create_Custom_model((31, 31, 1), 15)

CustomModel_128.compile(optimizer=SGD(learning_rate=learning_rate, momentum=momentum), loss='categorical_crossentropy', metrics=['accuracy',f1_m])
CustomModel_31.compile(optimizer=SGD(learning_rate=learning_rate, momentum=momentum), loss='categorical_crossentropy', metrics=['accuracy',f1_m])

CustomModel_128.summary()

Model: "model"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 input_1 (InputLayer)        [(None, 128, 128, 1)]     0         
                                                                 
 conv2d (Conv2D)             (None, 128, 128, 16)      160       
                                                                 
 batch_normalization (BatchN  (None, 128, 128, 16)     64        
 ormalization)                                                   
                                                                 
 activation (Activation)     (None, 128, 128, 16)      0         
                                                                 
 conv2d_1 (Conv2D)           (None, 128, 128, 16)      2320      
                                                                 
 batch_normalization_1 (Batc  (None, 128, 128, 16)     64        
 hNormalization)                                                 
                                                                 
 activation_1 (Activation)   (None, 128, 128, 16)      0         
                                                                 
 max_pooling2d (MaxPooling2D  (None, 64, 64, 16)       0         
 )                                                               
                                                                 
 conv2d_2 (Conv2D)           (None, 64, 64, 32)        4640      
                                                                 
 batch_normalization_2 (Batc  (None, 64, 64, 32)       128       
 hNormalization)                                                 
                                                                 
 activation_2 (Activation)   (None, 64, 64, 32)        0         
                                                                 
 conv2d_3 (Conv2D)           (None, 64, 64, 32)        9248      
                                                                 
 batch_normalization_3 (Batc  (None, 64, 64, 32)       128       
 hNormalization)                                                 
                                                                 
 activation_3 (Activation)   (None, 64, 64, 32)        0         
                                                                 
 max_pooling2d_1 (MaxPooling  (None, 32, 32, 32)       0         
 2D)                                                             
                                                                 
 conv2d_4 (Conv2D)           (None, 32, 32, 64)        18496     
                                                                 
 batch_normalization_4 (Batc  (None, 32, 32, 64)       256       
 hNormalization)                                                 
                                                                 
 activation_4 (Activation)   (None, 32, 32, 64)        0         
                                                                 
 conv2d_5 (Conv2D)           (None, 32, 32, 64)        36928     
                                                                 
 batch_normalization_5 (Batc  (None, 32, 32, 64)       256       
 hNormalization)                                                 
                                                                 
 activation_5 (Activation)   (None, 32, 32, 64)        0         
                                                                 
 max_pooling2d_2 (MaxPooling  (None, 16, 16, 64)       0         
 2D)                                                             
                                                                 
 conv2d_6 (Conv2D)           (None, 16, 16, 128)       73856     
                                                                 
 batch_normalization_6 (Batc  (None, 16, 16, 128)      512       
 hNormalization)                                                 
                                                                 
 activation_6 (Activation)   (None, 16, 16, 128)       0         
                                                                 
 conv2d_7 (Conv2D)           (None, 16, 16, 128)       147584    
                                                                 
 batch_normalization_7 (Batc  (None, 16, 16, 128)      512       
 hNormalization)                                                 
                                                                 
 activation_7 (Activation)   (None, 16, 16, 128)       0         
                                                                 
 conv2d_8 (Conv2D)           (None, 16, 16, 128)       147584    
                                                                 
 batch_normalization_8 (Batc  (None, 16, 16, 128)      512       
 hNormalization)                                                 
                                                                 
 activation_8 (Activation)   (None, 16, 16, 128)       0         
                                                                 
 max_pooling2d_3 (MaxPooling  (None, 8, 8, 128)        0         
 2D)                                                             
                                                                 
 conv2d_9 (Conv2D)           (None, 8, 8, 256)         295168    
                                                                 
 batch_normalization_9 (Batc  (None, 8, 8, 256)        1024      
 hNormalization)                                                 
                                                                 
 activation_9 (Activation)   (None, 8, 8, 256)         0         
                                                                 
 conv2d_10 (Conv2D)          (None, 8, 8, 256)         590080    
                                                                 
 batch_normalization_10 (Bat  (None, 8, 8, 256)        1024      
 chNormalization)                                                
                                                                 
 activation_10 (Activation)  (None, 8, 8, 256)         0         
                                                                 
 conv2d_11 (Conv2D)          (None, 8, 8, 256)         590080    
                                                                 
 batch_normalization_11 (Bat  (None, 8, 8, 256)        1024      
 chNormalization)                                                
                                                                 
 activation_11 (Activation)  (None, 8, 8, 256)         0         
                                                                 
 max_pooling2d_4 (MaxPooling  (None, 4, 4, 256)        0         
 2D)                                                             
                                                                 
 flatten (Flatten)           (None, 4096)              0         
                                                                 
 dense (Dense)               (None, 256)               1048832   
                                                                 
 dense_1 (Dense)             (None, 256)               65792     
                                                                 
 dropout (Dropout)           (None, 256)               0         
                                                                 
 dense_2 (Dense)             (None, 15)                3855      
                                                                 
=================================================================
Total params: 3,040,127
Trainable params: 3,037,375
Non-trainable params: 2,752
_________________________________________________________________

CustomModel_31.summary()

Model: "model"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 input_1 (InputLayer)        [(None, 31, 31, 1)]       0         
                                                                 
 conv2d (Conv2D)             (None, 31, 31, 64)        640       
                                                                 
 batch_normalization (BatchN  (None, 31, 31, 64)       256       
 ormalization)                                                   
                                                                 
 activation (Activation)     (None, 31, 31, 64)        0         
                                                                 
 conv2d_1 (Conv2D)           (None, 31, 31, 64)        36928     
                                                                 
 batch_normalization_1 (Batc  (None, 31, 31, 64)       256       
 hNormalization)                                                 
                                                                 
 activation_1 (Activation)   (None, 31, 31, 64)        0         
                                                                 
 conv2d_2 (Conv2D)           (None, 31, 31, 64)        36928     
                                                                 
 batch_normalization_2 (Batc  (None, 31, 31, 64)       256       
 hNormalization)                                                 
                                                                 
 activation_2 (Activation)   (None, 31, 31, 64)        0         
                                                                 
 max_pooling2d (MaxPooling2D  (None, 15, 15, 64)       0         
 )                                                               
                                                                 
 conv2d_3 (Conv2D)           (None, 15, 15, 128)       73856     
                                                                 
 batch_normalization_3 (Batc  (None, 15, 15, 128)      512       
 hNormalization)                                                 
                                                                 
 activation_3 (Activation)   (None, 15, 15, 128)       0         
                                                                 
 conv2d_4 (Conv2D)           (None, 15, 15, 128)       147584    
                                                                 
 batch_normalization_4 (Batc  (None, 15, 15, 128)      512       
 hNormalization)                                                 
                                                                 
 activation_4 (Activation)   (None, 15, 15, 128)       0         
                                                                 
 conv2d_5 (Conv2D)           (None, 15, 15, 128)       147584    
                                                                 
 batch_normalization_5 (Batc  (None, 15, 15, 128)      512       
 hNormalization)                                                 
                                                                 
 activation_5 (Activation)   (None, 15, 15, 128)       0         
                                                                 
 max_pooling2d_1 (MaxPooling  (None, 7, 7, 128)        0         
 2D)                                                             
                                                                 
 conv2d_6 (Conv2D)           (None, 7, 7, 256)         295168    
                                                                 
 batch_normalization_6 (Batc  (None, 7, 7, 256)        1024      
 hNormalization)                                                 
                                                                 
 activation_6 (Activation)   (None, 7, 7, 256)         0         
                                                                 
 conv2d_7 (Conv2D)           (None, 7, 7, 256)         590080    
                                                                 
 batch_normalization_7 (Batc  (None, 7, 7, 256)        1024      
 hNormalization)                                                 
                                                                 
 activation_7 (Activation)   (None, 7, 7, 256)         0         
                                                                 
 conv2d_8 (Conv2D)           (None, 7, 7, 256)         590080    
                                                                 
 batch_normalization_8 (Batc  (None, 7, 7, 256)        1024      
 hNormalization)                                                 
                                                                 
 activation_8 (Activation)   (None, 7, 7, 256)         0         
                                                                 
 max_pooling2d_2 (MaxPooling  (None, 3, 3, 256)        0         
 2D)                                                             
                                                                 
 flatten (Flatten)           (None, 2304)              0         
                                                                 
 dense (Dense)               (None, 256)               590080    
                                                                 
 dense_1 (Dense)             (None, 256)               65792     
                                                                 
 dropout (Dropout)           (None, 256)               0         
                                                                 
 dense_2 (Dense)             (None, 15)                3855      
                                                                 
=================================================================
Total params: 2,583,951
Trainable params: 2,581,263
Non-trainable params: 2,688
_________________________________________________________________

Training without Data Augmentation

%%time
CustomModel_128.fit(dataset_train_128, epochs=150, validation_data=dataset_test_128, class_weight=class_weights_dict,callbacks=[EarlyStopping(monitor='val_loss', patience=10, verbose=1, mode='min')],steps_per_epoch=steps_per_epoch)

Epoch 1/150
142/142 [==============================] - 10s 62ms/step - loss: 0.4669 - accuracy: 0.3795 - f1_m: 0.2279 - val_loss: 3.0552 - val_accuracy: 0.1193 - val_f1_m: 0.0000e+00
Epoch 2/150
142/142 [==============================] - 8s 59ms/step - loss: 0.3966 - accuracy: 0.6915 - f1_m: 0.6550 - val_loss: 2.0017 - val_accuracy: 0.4380 - val_f1_m: 0.3427
Epoch 3/150
142/142 [==============================] - 8s 59ms/step - loss: 0.3518 - accuracy: 0.8412 - f1_m: 0.8301 - val_loss: 1.5338 - val_accuracy: 0.6123 - val_f1_m: 0.5956
Epoch 4/150
142/142 [==============================] - 8s 59ms/step - loss: 0.3306 - accuracy: 0.8636 - f1_m: 0.8607 - val_loss: 1.2626 - val_accuracy: 0.7280 - val_f1_m: 0.7283
Epoch 5/150
142/142 [==============================] - 8s 59ms/step - loss: 0.3033 - accuracy: 0.9289 - f1_m: 0.9253 - val_loss: 0.9722 - val_accuracy: 0.7707 - val_f1_m: 0.7762
Epoch 6/150
142/142 [==============================] - 8s 59ms/step - loss: 0.2824 - accuracy: 0.9613 - f1_m: 0.9593 - val_loss: 0.5643 - val_accuracy: 0.9050 - val_f1_m: 0.9072
Epoch 7/150
142/142 [==============================] - 8s 59ms/step - loss: 0.2655 - accuracy: 0.9743 - f1_m: 0.9741 - val_loss: 0.4851 - val_accuracy: 0.9273 - val_f1_m: 0.9290
Epoch 8/150
142/142 [==============================] - 8s 59ms/step - loss: 0.2496 - accuracy: 0.9857 - f1_m: 0.9800 - val_loss: 0.7654 - val_accuracy: 0.8257 - val_f1_m: 0.8269
Epoch 9/150
142/142 [==============================] - 8s 59ms/step - loss: 0.2410 - accuracy: 0.9602 - f1_m: 0.9589 - val_loss: 0.6492 - val_accuracy: 0.8690 - val_f1_m: 0.8744
Epoch 10/150
142/142 [==============================] - 8s 59ms/step - loss: 0.2239 - accuracy: 0.9855 - f1_m: 0.9849 - val_loss: 0.4848 - val_accuracy: 0.9220 - val_f1_m: 0.9243
Epoch 11/150
142/142 [==============================] - 8s 59ms/step - loss: 0.2101 - accuracy: 0.9962 - f1_m: 0.9929 - val_loss: 0.3863 - val_accuracy: 0.9477 - val_f1_m: 0.9478
Epoch 12/150
142/142 [==============================] - 8s 59ms/step - loss: 0.1998 - accuracy: 0.9894 - f1_m: 0.9889 - val_loss: 0.3933 - val_accuracy: 0.9417 - val_f1_m: 0.9419
Epoch 13/150
142/142 [==============================] - 9s 59ms/step - loss: 0.1883 - accuracy: 0.9955 - f1_m: 0.9955 - val_loss: 0.3361 - val_accuracy: 0.9540 - val_f1_m: 0.9544
Epoch 14/150
142/142 [==============================] - 8s 59ms/step - loss: 0.1776 - accuracy: 0.9979 - f1_m: 0.9978 - val_loss: 0.3102 - val_accuracy: 0.9610 - val_f1_m: 0.9620
Epoch 15/150
142/142 [==============================] - 8s 59ms/step - loss: 0.1684 - accuracy: 0.9968 - f1_m: 0.9958 - val_loss: 0.2970 - val_accuracy: 0.9630 - val_f1_m: 0.9644
Epoch 16/150
142/142 [==============================] - 8s 59ms/step - loss: 0.1593 - accuracy: 0.9971 - f1_m: 0.9942 - val_loss: 0.6286 - val_accuracy: 0.8467 - val_f1_m: 0.8490
Epoch 17/150
142/142 [==============================] - 8s 59ms/step - loss: 0.1631 - accuracy: 0.9373 - f1_m: 0.9307 - val_loss: 0.5175 - val_accuracy: 0.8817 - val_f1_m: 0.8853
Epoch 18/150
142/142 [==============================] - 8s 59ms/step - loss: 0.1473 - accuracy: 0.9771 - f1_m: 0.9752 - val_loss: 0.2817 - val_accuracy: 0.9577 - val_f1_m: 0.9583
Epoch 19/150
142/142 [==============================] - 9s 60ms/step - loss: 0.1360 - accuracy: 0.9969 - f1_m: 0.9969 - val_loss: 0.3471 - val_accuracy: 0.9307 - val_f1_m: 0.9295
Epoch 20/150
142/142 [==============================] - 8s 59ms/step - loss: 0.1309 - accuracy: 0.9884 - f1_m: 0.9883 - val_loss: 0.2853 - val_accuracy: 0.9527 - val_f1_m: 0.9552
Epoch 21/150
142/142 [==============================] - 8s 59ms/step - loss: 0.1216 - accuracy: 0.9981 - f1_m: 0.9981 - val_loss: 0.2560 - val_accuracy: 0.9610 - val_f1_m: 0.9619
Epoch 22/150
142/142 [==============================] - 8s 59ms/step - loss: 0.1150 - accuracy: 0.9987 - f1_m: 0.9974 - val_loss: 0.2733 - val_accuracy: 0.9533 - val_f1_m: 0.9556
Epoch 23/150
142/142 [==============================] - 8s 59ms/step - loss: 0.1096 - accuracy: 0.9951 - f1_m: 0.9939 - val_loss: 0.2129 - val_accuracy: 0.9687 - val_f1_m: 0.9700
Epoch 24/150
142/142 [==============================] - 8s 59ms/step - loss: 0.1032 - accuracy: 0.9990 - f1_m: 0.9960 - val_loss: 0.2399 - val_accuracy: 0.9597 - val_f1_m: 0.9589
Epoch 25/150
142/142 [==============================] - 9s 66ms/step - loss: 0.0982 - accuracy: 0.9960 - f1_m: 0.9949 - val_loss: 0.2139 - val_accuracy: 0.9650 - val_f1_m: 0.9657
Epoch 26/150
142/142 [==============================] - 9s 61ms/step - loss: 0.0928 - accuracy: 0.9983 - f1_m: 0.9980 - val_loss: 0.2078 - val_accuracy: 0.9647 - val_f1_m: 0.9651
Epoch 27/150
142/142 [==============================] - 8s 59ms/step - loss: 0.0877 - accuracy: 0.9991 - f1_m: 0.9989 - val_loss: 0.1876 - val_accuracy: 0.9687 - val_f1_m: 0.9703
Epoch 28/150
142/142 [==============================] - 8s 59ms/step - loss: 0.0829 - accuracy: 0.9997 - f1_m: 0.9996 - val_loss: 0.1806 - val_accuracy: 0.9717 - val_f1_m: 0.9722
Epoch 29/150
142/142 [==============================] - 8s 59ms/step - loss: 0.0787 - accuracy: 0.9990 - f1_m: 0.9970 - val_loss: 0.1878 - val_accuracy: 0.9693 - val_f1_m: 0.9698
Epoch 30/150
142/142 [==============================] - 8s 59ms/step - loss: 0.0747 - accuracy: 0.9981 - f1_m: 0.9978 - val_loss: 0.1700 - val_accuracy: 0.9717 - val_f1_m: 0.9734
Epoch 31/150
142/142 [==============================] - 8s 59ms/step - loss: 0.0706 - accuracy: 0.9998 - f1_m: 0.9997 - val_loss: 0.1645 - val_accuracy: 0.9737 - val_f1_m: 0.9743
Epoch 32/150
142/142 [==============================] - 8s 59ms/step - loss: 0.0668 - accuracy: 0.9998 - f1_m: 0.9981 - val_loss: 0.2878 - val_accuracy: 0.9340 - val_f1_m: 0.9340
Epoch 33/150
142/142 [==============================] - 8s 59ms/step - loss: 0.0676 - accuracy: 0.9816 - f1_m: 0.9809 - val_loss: 0.1885 - val_accuracy: 0.9643 - val_f1_m: 0.9644
Epoch 34/150
142/142 [==============================] - 8s 59ms/step - loss: 0.0605 - accuracy: 0.9994 - f1_m: 0.9986 - val_loss: 0.1777 - val_accuracy: 0.9687 - val_f1_m: 0.9691
Epoch 35/150
142/142 [==============================] - 8s 59ms/step - loss: 0.0577 - accuracy: 0.9983 - f1_m: 0.9978 - val_loss: 0.1959 - val_accuracy: 0.9617 - val_f1_m: 0.9626
Epoch 36/150
142/142 [==============================] - 8s 59ms/step - loss: 0.0547 - accuracy: 0.9991 - f1_m: 0.9991 - val_loss: 0.1696 - val_accuracy: 0.9663 - val_f1_m: 0.9670
Epoch 37/150
142/142 [==============================] - 8s 59ms/step - loss: 0.0516 - accuracy: 0.9998 - f1_m: 0.9986 - val_loss: 0.1672 - val_accuracy: 0.9687 - val_f1_m: 0.9691
Epoch 38/150
142/142 [==============================] - 8s 59ms/step - loss: 0.0495 - accuracy: 0.9980 - f1_m: 0.9977 - val_loss: 0.1472 - val_accuracy: 0.9717 - val_f1_m: 0.9730
Epoch 39/150
142/142 [==============================] - 8s 59ms/step - loss: 0.0467 - accuracy: 0.9994 - f1_m: 0.9972 - val_loss: 0.1489 - val_accuracy: 0.9713 - val_f1_m: 0.9721
Epoch 40/150
142/142 [==============================] - 8s 59ms/step - loss: 0.0447 - accuracy: 0.9980 - f1_m: 0.9978 - val_loss: 0.1435 - val_accuracy: 0.9710 - val_f1_m: 0.9714
Epoch 41/150
142/142 [==============================] - 8s 59ms/step - loss: 0.0422 - accuracy: 0.9992 - f1_m: 0.9991 - val_loss: 0.1385 - val_accuracy: 0.9700 - val_f1_m: 0.9729
Epoch 42/150
142/142 [==============================] - 8s 59ms/step - loss: 0.0399 - accuracy: 1.0000 - f1_m: 0.9999 - val_loss: 0.1366 - val_accuracy: 0.9710 - val_f1_m: 0.9725
Epoch 43/150
142/142 [==============================] - 8s 59ms/step - loss: 0.0380 - accuracy: 0.9997 - f1_m: 0.9947 - val_loss: 0.7156 - val_accuracy: 0.8090 - val_f1_m: 0.8146
Epoch 44/150
142/142 [==============================] - 8s 59ms/step - loss: 0.0476 - accuracy: 0.9575 - f1_m: 0.9561 - val_loss: 0.2557 - val_accuracy: 0.9373 - val_f1_m: 0.9338
Epoch 45/150
142/142 [==============================] - 8s 59ms/step - loss: 0.0355 - accuracy: 0.9983 - f1_m: 0.9981 - val_loss: 0.1423 - val_accuracy: 0.9697 - val_f1_m: 0.9688
Epoch 46/150
142/142 [==============================] - 8s 59ms/step - loss: 0.0334 - accuracy: 0.9987 - f1_m: 0.9986 - val_loss: 0.1372 - val_accuracy: 0.9697 - val_f1_m: 0.9709
Epoch 47/150
142/142 [==============================] - 8s 59ms/step - loss: 0.0322 - accuracy: 0.9970 - f1_m: 0.9969 - val_loss: 0.1377 - val_accuracy: 0.9697 - val_f1_m: 0.9694
Epoch 48/150
142/142 [==============================] - 8s 59ms/step - loss: 0.0301 - accuracy: 0.9996 - f1_m: 0.9993 - val_loss: 0.1282 - val_accuracy: 0.9740 - val_f1_m: 0.9729
Epoch 49/150
142/142 [==============================] - 8s 59ms/step - loss: 0.0288 - accuracy: 0.9990 - f1_m: 0.9986 - val_loss: 0.1414 - val_accuracy: 0.9640 - val_f1_m: 0.9651
Epoch 50/150
142/142 [==============================] - 8s 59ms/step - loss: 0.0275 - accuracy: 0.9986 - f1_m: 0.9986 - val_loss: 0.1217 - val_accuracy: 0.9737 - val_f1_m: 0.9736
Epoch 51/150
142/142 [==============================] - 8s 59ms/step - loss: 0.0260 - accuracy: 0.9997 - f1_m: 0.9979 - val_loss: 0.1963 - val_accuracy: 0.9473 - val_f1_m: 0.9490
Epoch 52/150
142/142 [==============================] - 8s 59ms/step - loss: 0.0297 - accuracy: 0.9775 - f1_m: 0.9786 - val_loss: 0.1271 - val_accuracy: 0.9723 - val_f1_m: 0.9713
Epoch 53/150
142/142 [==============================] - 8s 59ms/step - loss: 0.0239 - accuracy: 0.9994 - f1_m: 0.9993 - val_loss: 0.1167 - val_accuracy: 0.9740 - val_f1_m: 0.9743
Epoch 54/150
142/142 [==============================] - 8s 59ms/step - loss: 0.0228 - accuracy: 0.9991 - f1_m: 0.9992 - val_loss: 0.1115 - val_accuracy: 0.9747 - val_f1_m: 0.9750
Epoch 55/150
142/142 [==============================] - 8s 59ms/step - loss: 0.0215 - accuracy: 0.9998 - f1_m: 0.9988 - val_loss: 0.1167 - val_accuracy: 0.9717 - val_f1_m: 0.9731
Epoch 56/150
142/142 [==============================] - 8s 59ms/step - loss: 0.0209 - accuracy: 0.9991 - f1_m: 0.9990 - val_loss: 0.1020 - val_accuracy: 0.9773 - val_f1_m: 0.9775
Epoch 57/150
142/142 [==============================] - 8s 59ms/step - loss: 0.0197 - accuracy: 1.0000 - f1_m: 1.0000 - val_loss: 0.1050 - val_accuracy: 0.9777 - val_f1_m: 0.9785
Epoch 58/150
142/142 [==============================] - 8s 59ms/step - loss: 0.0187 - accuracy: 1.0000 - f1_m: 0.9999 - val_loss: 0.0965 - val_accuracy: 0.9790 - val_f1_m: 0.9799
Epoch 59/150
142/142 [==============================] - 8s 59ms/step - loss: 0.0179 - accuracy: 0.9997 - f1_m: 0.9981 - val_loss: 0.1027 - val_accuracy: 0.9780 - val_f1_m: 0.9763
Epoch 60/150
142/142 [==============================] - 8s 59ms/step - loss: 0.0177 - accuracy: 0.9979 - f1_m: 0.9969 - val_loss: 0.1345 - val_accuracy: 0.9650 - val_f1_m: 0.9656
Epoch 61/150
142/142 [==============================] - 8s 59ms/step - loss: 0.0166 - accuracy: 0.9990 - f1_m: 0.9992 - val_loss: 0.0969 - val_accuracy: 0.9780 - val_f1_m: 0.9790
Epoch 62/150
142/142 [==============================] - 8s 58ms/step - loss: 0.0159 - accuracy: 0.9994 - f1_m: 0.9978 - val_loss: 0.4235 - val_accuracy: 0.8777 - val_f1_m: 0.8775
Epoch 63/150
142/142 [==============================] - 8s 59ms/step - loss: 0.0244 - accuracy: 0.9744 - f1_m: 0.9694 - val_loss: 0.1809 - val_accuracy: 0.9510 - val_f1_m: 0.9535
Epoch 64/150
142/142 [==============================] - 8s 59ms/step - loss: 0.0163 - accuracy: 0.9944 - f1_m: 0.9943 - val_loss: 0.1207 - val_accuracy: 0.9737 - val_f1_m: 0.9725
Epoch 65/150
142/142 [==============================] - 8s 59ms/step - loss: 0.0145 - accuracy: 0.9992 - f1_m: 0.9983 - val_loss: 0.1060 - val_accuracy: 0.9743 - val_f1_m: 0.9746
Epoch 66/150
142/142 [==============================] - 8s 59ms/step - loss: 0.0143 - accuracy: 0.9971 - f1_m: 0.9963 - val_loss: 0.1128 - val_accuracy: 0.9707 - val_f1_m: 0.9699
Epoch 67/150
142/142 [==============================] - 8s 59ms/step - loss: 0.0139 - accuracy: 0.9971 - f1_m: 0.9972 - val_loss: 0.0992 - val_accuracy: 0.9740 - val_f1_m: 0.9747
Epoch 68/150
142/142 [==============================] - 8s 59ms/step - loss: 0.0126 - accuracy: 0.9997 - f1_m: 0.9968 - val_loss: 0.1447 - val_accuracy: 0.9603 - val_f1_m: 0.9616
Epoch 68: early stopping
CPU times: total: 4min 22s
Wall time: 9min 37s

<keras.callbacks.History at 0x1de2eab9670>

%%time
CustomModel_31.fit(dataset_train_31, epochs=150, validation_data=dataset_test_31, class_weight=class_weights_dict,callbacks=[EarlyStopping(monitor='val_loss', patience=10, verbose=1, mode='min')],steps_per_epoch=steps_per_epoch)

Epoch 1/150
142/142 [==============================] - 5s 25ms/step - loss: 0.4586 - accuracy: 0.3817 - f1_m: 0.2173 - val_loss: 2.7801 - val_accuracy: 0.1723 - val_f1_m: 6.5466e-04
Epoch 2/150
142/142 [==============================] - 3s 24ms/step - loss: 0.3886 - accuracy: 0.6776 - f1_m: 0.6218 - val_loss: 1.4081 - val_accuracy: 0.6863 - val_f1_m: 0.5629
Epoch 3/150
142/142 [==============================] - 3s 24ms/step - loss: 0.3471 - accuracy: 0.8173 - f1_m: 0.8037 - val_loss: 1.7350 - val_accuracy: 0.5420 - val_f1_m: 0.5242
Epoch 4/150
142/142 [==============================] - 3s 24ms/step - loss: 0.3170 - accuracy: 0.8979 - f1_m: 0.8932 - val_loss: 1.1168 - val_accuracy: 0.7330 - val_f1_m: 0.7269
Epoch 5/150
142/142 [==============================] - 3s 24ms/step - loss: 0.2973 - accuracy: 0.9195 - f1_m: 0.9149 - val_loss: 0.7898 - val_accuracy: 0.8397 - val_f1_m: 0.8405
Epoch 6/150
142/142 [==============================] - 3s 24ms/step - loss: 0.2752 - accuracy: 0.9662 - f1_m: 0.9643 - val_loss: 0.7310 - val_accuracy: 0.8570 - val_f1_m: 0.8580
Epoch 7/150
142/142 [==============================] - 3s 24ms/step - loss: 0.2584 - accuracy: 0.9826 - f1_m: 0.9795 - val_loss: 0.6288 - val_accuracy: 0.8923 - val_f1_m: 0.8927
Epoch 8/150
142/142 [==============================] - 3s 24ms/step - loss: 0.2434 - accuracy: 0.9900 - f1_m: 0.9887 - val_loss: 1.1280 - val_accuracy: 0.7337 - val_f1_m: 0.7297
Epoch 9/150
142/142 [==============================] - 3s 24ms/step - loss: 0.2316 - accuracy: 0.9850 - f1_m: 0.9829 - val_loss: 0.5520 - val_accuracy: 0.8997 - val_f1_m: 0.8979
Epoch 10/150
142/142 [==============================] - 3s 24ms/step - loss: 0.2175 - accuracy: 0.9940 - f1_m: 0.9933 - val_loss: 0.5454 - val_accuracy: 0.8983 - val_f1_m: 0.9011
Epoch 11/150
142/142 [==============================] - 3s 24ms/step - loss: 0.2050 - accuracy: 0.9977 - f1_m: 0.9975 - val_loss: 0.5860 - val_accuracy: 0.8773 - val_f1_m: 0.8781
Epoch 12/150
142/142 [==============================] - 3s 24ms/step - loss: 0.1936 - accuracy: 0.9992 - f1_m: 0.9946 - val_loss: 0.7436 - val_accuracy: 0.8263 - val_f1_m: 0.8287
Epoch 13/150
142/142 [==============================] - 3s 24ms/step - loss: 0.1871 - accuracy: 0.9764 - f1_m: 0.9750 - val_loss: 0.5195 - val_accuracy: 0.8900 - val_f1_m: 0.8912
Epoch 14/150
142/142 [==============================] - 3s 24ms/step - loss: 0.1741 - accuracy: 0.9979 - f1_m: 0.9930 - val_loss: 0.7036 - val_accuracy: 0.8413 - val_f1_m: 0.8470
Epoch 15/150
142/142 [==============================] - 3s 23ms/step - loss: 0.1699 - accuracy: 0.9767 - f1_m: 0.9728 - val_loss: 0.7253 - val_accuracy: 0.8210 - val_f1_m: 0.8209
Epoch 16/150
142/142 [==============================] - 3s 24ms/step - loss: 0.1597 - accuracy: 0.9785 - f1_m: 0.9782 - val_loss: 0.5455 - val_accuracy: 0.8817 - val_f1_m: 0.8815
Epoch 17/150
142/142 [==============================] - 3s 24ms/step - loss: 0.1478 - accuracy: 0.9992 - f1_m: 0.9981 - val_loss: 0.4442 - val_accuracy: 0.9120 - val_f1_m: 0.9135
Epoch 18/150
142/142 [==============================] - 3s 24ms/step - loss: 0.1404 - accuracy: 0.9966 - f1_m: 0.9951 - val_loss: 0.4370 - val_accuracy: 0.9103 - val_f1_m: 0.9088
Epoch 19/150
142/142 [==============================] - 3s 24ms/step - loss: 0.1331 - accuracy: 0.9960 - f1_m: 0.9956 - val_loss: 0.3994 - val_accuracy: 0.9227 - val_f1_m: 0.9248
Epoch 20/150
142/142 [==============================] - 3s 24ms/step - loss: 0.1259 - accuracy: 0.9982 - f1_m: 0.9978 - val_loss: 0.3806 - val_accuracy: 0.9247 - val_f1_m: 0.9299
Epoch 21/150
142/142 [==============================] - 3s 24ms/step - loss: 0.1186 - accuracy: 0.9997 - f1_m: 0.9997 - val_loss: 0.3743 - val_accuracy: 0.9233 - val_f1_m: 0.9254
Epoch 22/150
142/142 [==============================] - 3s 24ms/step - loss: 0.1124 - accuracy: 0.9990 - f1_m: 0.9990 - val_loss: 0.3743 - val_accuracy: 0.9237 - val_f1_m: 0.9249
Epoch 23/150
142/142 [==============================] - 3s 23ms/step - loss: 0.1066 - accuracy: 0.9984 - f1_m: 0.9919 - val_loss: 0.6022 - val_accuracy: 0.8473 - val_f1_m: 0.8484
Epoch 24/150
142/142 [==============================] - 3s 23ms/step - loss: 0.1040 - accuracy: 0.9832 - f1_m: 0.9811 - val_loss: 0.4843 - val_accuracy: 0.8833 - val_f1_m: 0.8854
Epoch 25/150
142/142 [==============================] - 4s 25ms/step - loss: 0.0985 - accuracy: 0.9894 - f1_m: 0.9896 - val_loss: 0.3382 - val_accuracy: 0.9270 - val_f1_m: 0.9302
Epoch 26/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0914 - accuracy: 0.9981 - f1_m: 0.9964 - val_loss: 0.3607 - val_accuracy: 0.9237 - val_f1_m: 0.9233
Epoch 27/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0865 - accuracy: 0.9979 - f1_m: 0.9979 - val_loss: 0.3383 - val_accuracy: 0.9280 - val_f1_m: 0.9290
Epoch 28/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0816 - accuracy: 0.9999 - f1_m: 0.9999 - val_loss: 0.3276 - val_accuracy: 0.9273 - val_f1_m: 0.9292
Epoch 29/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0774 - accuracy: 0.9999 - f1_m: 0.9982 - val_loss: 0.3443 - val_accuracy: 0.9233 - val_f1_m: 0.9260
Epoch 30/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0742 - accuracy: 0.9973 - f1_m: 0.9971 - val_loss: 0.3122 - val_accuracy: 0.9320 - val_f1_m: 0.9314
Epoch 31/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0696 - accuracy: 0.9999 - f1_m: 0.9982 - val_loss: 0.3114 - val_accuracy: 0.9323 - val_f1_m: 0.9355
Epoch 32/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0684 - accuracy: 0.9915 - f1_m: 0.9881 - val_loss: 0.3542 - val_accuracy: 0.9127 - val_f1_m: 0.9141
Epoch 33/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0642 - accuracy: 0.9944 - f1_m: 0.9943 - val_loss: 0.2957 - val_accuracy: 0.9300 - val_f1_m: 0.9327
Epoch 34/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0597 - accuracy: 0.9999 - f1_m: 0.9982 - val_loss: 0.4158 - val_accuracy: 0.8953 - val_f1_m: 0.8955
Epoch 35/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0575 - accuracy: 0.9967 - f1_m: 0.9940 - val_loss: 0.3334 - val_accuracy: 0.9153 - val_f1_m: 0.9181
Epoch 36/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0567 - accuracy: 0.9899 - f1_m: 0.9893 - val_loss: 0.3118 - val_accuracy: 0.9210 - val_f1_m: 0.9248
Epoch 37/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0515 - accuracy: 0.9989 - f1_m: 0.9990 - val_loss: 0.2953 - val_accuracy: 0.9293 - val_f1_m: 0.9310
Epoch 38/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0486 - accuracy: 0.9999 - f1_m: 0.9998 - val_loss: 0.2776 - val_accuracy: 0.9313 - val_f1_m: 0.9342
Epoch 39/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0461 - accuracy: 1.0000 - f1_m: 1.0000 - val_loss: 0.2694 - val_accuracy: 0.9360 - val_f1_m: 0.9367
Epoch 40/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0439 - accuracy: 0.9998 - f1_m: 0.9977 - val_loss: 0.3048 - val_accuracy: 0.9213 - val_f1_m: 0.9215
Epoch 41/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0437 - accuracy: 0.9927 - f1_m: 0.9894 - val_loss: 0.3970 - val_accuracy: 0.8950 - val_f1_m: 0.8937
Epoch 42/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0417 - accuracy: 0.9942 - f1_m: 0.9943 - val_loss: 0.2669 - val_accuracy: 0.9387 - val_f1_m: 0.9368
Epoch 43/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0381 - accuracy: 0.9999 - f1_m: 0.9990 - val_loss: 0.2706 - val_accuracy: 0.9337 - val_f1_m: 0.9363
Epoch 44/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0363 - accuracy: 0.9991 - f1_m: 0.9991 - val_loss: 0.2544 - val_accuracy: 0.9407 - val_f1_m: 0.9397
Epoch 45/150
142/142 [==============================] - 3s 24ms/step - loss: 0.0344 - accuracy: 0.9999 - f1_m: 0.9999 - val_loss: 0.2587 - val_accuracy: 0.9330 - val_f1_m: 0.9341
Epoch 46/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0329 - accuracy: 0.9997 - f1_m: 0.9986 - val_loss: 0.2786 - val_accuracy: 0.9300 - val_f1_m: 0.9327
Epoch 47/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0327 - accuracy: 0.9947 - f1_m: 0.9942 - val_loss: 0.2613 - val_accuracy: 0.9357 - val_f1_m: 0.9349
Epoch 48/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0299 - accuracy: 0.9998 - f1_m: 0.9998 - val_loss: 0.2326 - val_accuracy: 0.9433 - val_f1_m: 0.9450
Epoch 49/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0285 - accuracy: 0.9999 - f1_m: 0.9998 - val_loss: 0.2394 - val_accuracy: 0.9410 - val_f1_m: 0.9417
Epoch 50/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0271 - accuracy: 0.9999 - f1_m: 0.9999 - val_loss: 0.2336 - val_accuracy: 0.9430 - val_f1_m: 0.9446
Epoch 51/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0262 - accuracy: 0.9986 - f1_m: 0.9963 - val_loss: 0.2763 - val_accuracy: 0.9273 - val_f1_m: 0.9255
Epoch 52/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0257 - accuracy: 0.9955 - f1_m: 0.9954 - val_loss: 0.2360 - val_accuracy: 0.9410 - val_f1_m: 0.9404
Epoch 53/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0238 - accuracy: 0.9993 - f1_m: 0.9993 - val_loss: 0.2284 - val_accuracy: 0.9413 - val_f1_m: 0.9422
Epoch 54/150
142/142 [==============================] - 3s 24ms/step - loss: 0.0226 - accuracy: 0.9999 - f1_m: 0.9999 - val_loss: 0.2246 - val_accuracy: 0.9427 - val_f1_m: 0.9437
Epoch 55/150
142/142 [==============================] - 3s 24ms/step - loss: 0.0216 - accuracy: 1.0000 - f1_m: 1.0000 - val_loss: 0.2219 - val_accuracy: 0.9423 - val_f1_m: 0.9439
Epoch 56/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0206 - accuracy: 0.9999 - f1_m: 0.9999 - val_loss: 0.2247 - val_accuracy: 0.9403 - val_f1_m: 0.9413
Epoch 57/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0197 - accuracy: 1.0000 - f1_m: 1.0000 - val_loss: 0.2410 - val_accuracy: 0.9367 - val_f1_m: 0.9372
Epoch 58/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0195 - accuracy: 0.9981 - f1_m: 0.9980 - val_loss: 0.2277 - val_accuracy: 0.9360 - val_f1_m: 0.9402
Epoch 59/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0182 - accuracy: 0.9998 - f1_m: 0.9989 - val_loss: 0.4779 - val_accuracy: 0.8540 - val_f1_m: 0.8541
Epoch 60/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0232 - accuracy: 0.9816 - f1_m: 0.9804 - val_loss: 0.2474 - val_accuracy: 0.9333 - val_f1_m: 0.9325
Epoch 61/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0171 - accuracy: 0.9998 - f1_m: 0.9997 - val_loss: 0.2183 - val_accuracy: 0.9427 - val_f1_m: 0.9441
Epoch 62/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0165 - accuracy: 0.9990 - f1_m: 0.9974 - val_loss: 0.2336 - val_accuracy: 0.9387 - val_f1_m: 0.9403
Epoch 63/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0166 - accuracy: 0.9956 - f1_m: 0.9954 - val_loss: 0.2129 - val_accuracy: 0.9447 - val_f1_m: 0.9440
Epoch 64/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0150 - accuracy: 1.0000 - f1_m: 0.9990 - val_loss: 0.2217 - val_accuracy: 0.9397 - val_f1_m: 0.9407
Epoch 65/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0150 - accuracy: 0.9984 - f1_m: 0.9980 - val_loss: 0.2110 - val_accuracy: 0.9440 - val_f1_m: 0.9466
Epoch 66/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0138 - accuracy: 1.0000 - f1_m: 0.9990 - val_loss: 0.2460 - val_accuracy: 0.9360 - val_f1_m: 0.9359
Epoch 67/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0145 - accuracy: 0.9950 - f1_m: 0.9928 - val_loss: 0.2473 - val_accuracy: 0.9343 - val_f1_m: 0.9350
Epoch 68/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0139 - accuracy: 0.9949 - f1_m: 0.9950 - val_loss: 0.2025 - val_accuracy: 0.9463 - val_f1_m: 0.9457
Epoch 69/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0124 - accuracy: 1.0000 - f1_m: 0.9990 - val_loss: 0.2485 - val_accuracy: 0.9287 - val_f1_m: 0.9297
Epoch 70/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0126 - accuracy: 0.9963 - f1_m: 0.9965 - val_loss: 0.2211 - val_accuracy: 0.9413 - val_f1_m: 0.9422
Epoch 71/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0119 - accuracy: 0.9988 - f1_m: 0.9972 - val_loss: 0.2137 - val_accuracy: 0.9427 - val_f1_m: 0.9437
Epoch 72/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0125 - accuracy: 0.9934 - f1_m: 0.9918 - val_loss: 0.2884 - val_accuracy: 0.9150 - val_f1_m: 0.9172
Epoch 73/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0131 - accuracy: 0.9911 - f1_m: 0.9906 - val_loss: 0.2271 - val_accuracy: 0.9407 - val_f1_m: 0.9413
Epoch 74/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0106 - accuracy: 1.0000 - f1_m: 1.0000 - val_loss: 0.1970 - val_accuracy: 0.9453 - val_f1_m: 0.9475
Epoch 75/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0102 - accuracy: 0.9999 - f1_m: 0.9999 - val_loss: 0.2141 - val_accuracy: 0.9417 - val_f1_m: 0.9409
Epoch 76/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0101 - accuracy: 0.9990 - f1_m: 0.9979 - val_loss: 0.2303 - val_accuracy: 0.9347 - val_f1_m: 0.9354
Epoch 77/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0105 - accuracy: 0.9978 - f1_m: 0.9970 - val_loss: 0.1946 - val_accuracy: 0.9477 - val_f1_m: 0.9475
Epoch 78/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0094 - accuracy: 0.9997 - f1_m: 0.9997 - val_loss: 0.2115 - val_accuracy: 0.9420 - val_f1_m: 0.9399
Epoch 79/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0091 - accuracy: 0.9998 - f1_m: 0.9981 - val_loss: 0.3952 - val_accuracy: 0.8877 - val_f1_m: 0.8911
Epoch 80/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0140 - accuracy: 0.9783 - f1_m: 0.9776 - val_loss: 0.2344 - val_accuracy: 0.9347 - val_f1_m: 0.9363
Epoch 81/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0089 - accuracy: 0.9993 - f1_m: 0.9993 - val_loss: 0.1938 - val_accuracy: 0.9490 - val_f1_m: 0.9508
Epoch 82/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0085 - accuracy: 0.9998 - f1_m: 0.9988 - val_loss: 0.2001 - val_accuracy: 0.9457 - val_f1_m: 0.9471
Epoch 83/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0086 - accuracy: 0.9984 - f1_m: 0.9981 - val_loss: 0.1901 - val_accuracy: 0.9483 - val_f1_m: 0.9495
Epoch 84/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0078 - accuracy: 1.0000 - f1_m: 1.0000 - val_loss: 0.1847 - val_accuracy: 0.9520 - val_f1_m: 0.9526
Epoch 85/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0076 - accuracy: 0.9998 - f1_m: 0.9998 - val_loss: 0.1855 - val_accuracy: 0.9503 - val_f1_m: 0.9517
Epoch 86/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0074 - accuracy: 0.9997 - f1_m: 0.9953 - val_loss: 0.3298 - val_accuracy: 0.9043 - val_f1_m: 0.9091
Epoch 87/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0091 - accuracy: 0.9921 - f1_m: 0.9897 - val_loss: 0.2782 - val_accuracy: 0.9193 - val_f1_m: 0.9195
Epoch 88/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0079 - accuracy: 0.9968 - f1_m: 0.9957 - val_loss: 0.2101 - val_accuracy: 0.9417 - val_f1_m: 0.9400
Epoch 89/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0073 - accuracy: 0.9987 - f1_m: 0.9986 - val_loss: 0.1921 - val_accuracy: 0.9483 - val_f1_m: 0.9476
Epoch 90/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0067 - accuracy: 1.0000 - f1_m: 1.0000 - val_loss: 0.1817 - val_accuracy: 0.9520 - val_f1_m: 0.9516
Epoch 91/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0065 - accuracy: 1.0000 - f1_m: 1.0000 - val_loss: 0.1799 - val_accuracy: 0.9503 - val_f1_m: 0.9510
Epoch 92/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0064 - accuracy: 1.0000 - f1_m: 1.0000 - val_loss: 0.1837 - val_accuracy: 0.9510 - val_f1_m: 0.9520
Epoch 93/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0062 - accuracy: 1.0000 - f1_m: 1.0000 - val_loss: 0.1857 - val_accuracy: 0.9470 - val_f1_m: 0.9475
Epoch 94/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0060 - accuracy: 1.0000 - f1_m: 0.9989 - val_loss: 0.2169 - val_accuracy: 0.9407 - val_f1_m: 0.9416
Epoch 95/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0069 - accuracy: 0.9966 - f1_m: 0.9965 - val_loss: 0.1867 - val_accuracy: 0.9473 - val_f1_m: 0.9502
Epoch 96/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0058 - accuracy: 1.0000 - f1_m: 1.0000 - val_loss: 0.1780 - val_accuracy: 0.9490 - val_f1_m: 0.9519
Epoch 97/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0057 - accuracy: 1.0000 - f1_m: 1.0000 - val_loss: 0.1736 - val_accuracy: 0.9513 - val_f1_m: 0.9549
Epoch 98/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0055 - accuracy: 0.9999 - f1_m: 0.9970 - val_loss: 0.1917 - val_accuracy: 0.9447 - val_f1_m: 0.9462
Epoch 99/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0063 - accuracy: 0.9972 - f1_m: 0.9971 - val_loss: 0.1884 - val_accuracy: 0.9457 - val_f1_m: 0.9503
Epoch 100/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0054 - accuracy: 1.0000 - f1_m: 0.9999 - val_loss: 0.1798 - val_accuracy: 0.9497 - val_f1_m: 0.9506
Epoch 101/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0052 - accuracy: 1.0000 - f1_m: 1.0000 - val_loss: 0.1764 - val_accuracy: 0.9493 - val_f1_m: 0.9531
Epoch 102/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0051 - accuracy: 1.0000 - f1_m: 1.0000 - val_loss: 0.1775 - val_accuracy: 0.9490 - val_f1_m: 0.9531
Epoch 103/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0050 - accuracy: 0.9999 - f1_m: 0.9970 - val_loss: 0.2009 - val_accuracy: 0.9430 - val_f1_m: 0.9470
Epoch 104/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0058 - accuracy: 0.9962 - f1_m: 0.9947 - val_loss: 0.2004 - val_accuracy: 0.9397 - val_f1_m: 0.9399
Epoch 105/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0058 - accuracy: 0.9958 - f1_m: 0.9949 - val_loss: 0.2038 - val_accuracy: 0.9430 - val_f1_m: 0.9430
Epoch 106/150
142/142 [==============================] - 3s 24ms/step - loss: 0.0052 - accuracy: 0.9990 - f1_m: 0.9988 - val_loss: 0.1835 - val_accuracy: 0.9473 - val_f1_m: 0.9488
Epoch 107/150
142/142 [==============================] - 3s 23ms/step - loss: 0.0048 - accuracy: 0.9999 - f1_m: 0.9999 - val_loss: 0.1764 - val_accuracy: 0.9493 - val_f1_m: 0.9526
Epoch 107: early stopping
CPU times: total: 2min 22s
Wall time: 6min 2s

<keras.callbacks.History at 0x1de2ee368b0>

evaluate_and_save_model(CustomModel_128, 'CustomModel_128', 64,save_models=False)
evaluate_and_save_model(CustomModel_31, 'CustomModel_31', 64,save_models=False)

Model Name            CustomModel_128
Epochs                             68
Batch Size                         64
Train Loss                   0.018694
Val Loss                     0.096466
Train Acc                         1.0
Val Acc                         0.979
[Train - Val] Acc               0.021
[Train - Val] Loss          -0.077772
Name: CustomModel_128, dtype: object
Model Name            CustomModel_31
Epochs                           107
Batch Size                        64
Train Loss                  0.007828
Val Loss                    0.184748
Train Acc                        1.0
Val Acc                        0.952
[Train - Val] Acc              0.048
[Train - Val] Loss          -0.17692
Name: CustomModel_31, dtype: object

customModel_history_128 = json.load(open('./models/CustomModel_128_history.json', 'r'))
customModel_history_31 = json.load(open('./models/CustomModel_31_history.json', 'r'))
compare_curves(customModel_history_31, customModel_history_128, '31x31', '128x128')

Observations:

• We can see that our efforts to prevent overfitting has paid off, with our models having a much better time generalising to our data.
• The presence of fluctuations in the accuracy and loss metrics could be attributed to the use of dropout layers. As dropout layers randomly drops a fraction of neurons during training.This introduces variability during our training,

Training with Data Augmentation

%%time
CustomModel_128_DA = create_Custom_model((128, 128, 1), 15)
CustomModel_128_DA.compile(optimizer=SGD(learning_rate=learning_rate, momentum=momentum), loss='categorical_crossentropy', metrics=['accuracy',f1_m])
CustomModel_128_DA.fit(train_generator_128, epochs=150, validation_data=dataset_test_128, class_weight=class_weights_dict,callbacks=[EarlyStopping(monitor='val_loss', patience=10, verbose=1, mode='min')],steps_per_epoch=steps_per_epoch)

Epoch 1/150
142/142 [==============================] - 10s 61ms/step - loss: 0.4727 - accuracy: 0.3467 - f1_m: 0.1923 - val_loss: 3.2902 - val_accuracy: 0.0703 - val_f1_m: 0.0000e+00
Epoch 2/150
142/142 [==============================] - 8s 58ms/step - loss: 0.4027 - accuracy: 0.6447 - f1_m: 0.5972 - val_loss: 1.7182 - val_accuracy: 0.5480 - val_f1_m: 0.4367
Epoch 3/150
142/142 [==============================] - 8s 58ms/step - loss: 0.3666 - accuracy: 0.7522 - f1_m: 0.7339 - val_loss: 1.3187 - val_accuracy: 0.6607 - val_f1_m: 0.6430
Epoch 4/150
142/142 [==============================] - 8s 58ms/step - loss: 0.3374 - accuracy: 0.8209 - f1_m: 0.8182 - val_loss: 1.0460 - val_accuracy: 0.7493 - val_f1_m: 0.7468
Epoch 5/150
142/142 [==============================] - 8s 58ms/step - loss: 0.3124 - accuracy: 0.8728 - f1_m: 0.8679 - val_loss: 2.9299 - val_accuracy: 0.3740 - val_f1_m: 0.3742
Epoch 6/150
142/142 [==============================] - 8s 58ms/step - loss: 0.2928 - accuracy: 0.8938 - f1_m: 0.8883 - val_loss: 1.2245 - val_accuracy: 0.7003 - val_f1_m: 0.6987
Epoch 7/150
142/142 [==============================] - 8s 58ms/step - loss: 0.2805 - accuracy: 0.8871 - f1_m: 0.8858 - val_loss: 1.2606 - val_accuracy: 0.6927 - val_f1_m: 0.7019
Epoch 8/150
142/142 [==============================] - 8s 58ms/step - loss: 0.2598 - accuracy: 0.9231 - f1_m: 0.9183 - val_loss: 0.7067 - val_accuracy: 0.8373 - val_f1_m: 0.8403
Epoch 9/150
142/142 [==============================] - 8s 58ms/step - loss: 0.2480 - accuracy: 0.9169 - f1_m: 0.9113 - val_loss: 0.8324 - val_accuracy: 0.8090 - val_f1_m: 0.8057
Epoch 10/150
142/142 [==============================] - 8s 58ms/step - loss: 0.2330 - accuracy: 0.9345 - f1_m: 0.9338 - val_loss: 0.4296 - val_accuracy: 0.9320 - val_f1_m: 0.9327
Epoch 11/150
142/142 [==============================] - 8s 58ms/step - loss: 0.2174 - accuracy: 0.9556 - f1_m: 0.9539 - val_loss: 1.0145 - val_accuracy: 0.7510 - val_f1_m: 0.7539
Epoch 12/150
142/142 [==============================] - 8s 58ms/step - loss: 0.2058 - accuracy: 0.9559 - f1_m: 0.9562 - val_loss: 2.5332 - val_accuracy: 0.5090 - val_f1_m: 0.5176
Epoch 13/150
142/142 [==============================] - 8s 58ms/step - loss: 0.1942 - accuracy: 0.9595 - f1_m: 0.9596 - val_loss: 0.4058 - val_accuracy: 0.9350 - val_f1_m: 0.9346
Epoch 14/150
142/142 [==============================] - 8s 58ms/step - loss: 0.1831 - accuracy: 0.9692 - f1_m: 0.9667 - val_loss: 0.5229 - val_accuracy: 0.8840 - val_f1_m: 0.8859
Epoch 15/150
142/142 [==============================] - 8s 58ms/step - loss: 0.1742 - accuracy: 0.9642 - f1_m: 0.9609 - val_loss: 0.5287 - val_accuracy: 0.8827 - val_f1_m: 0.8880
Epoch 16/150
142/142 [==============================] - 8s 58ms/step - loss: 0.1649 - accuracy: 0.9684 - f1_m: 0.9619 - val_loss: 0.3865 - val_accuracy: 0.9380 - val_f1_m: 0.9388
Epoch 17/150
142/142 [==============================] - 8s 58ms/step - loss: 0.1578 - accuracy: 0.9589 - f1_m: 0.9548 - val_loss: 0.5415 - val_accuracy: 0.8760 - val_f1_m: 0.8764
Epoch 18/150
142/142 [==============================] - 8s 58ms/step - loss: 0.1494 - accuracy: 0.9612 - f1_m: 0.9579 - val_loss: 0.7424 - val_accuracy: 0.8283 - val_f1_m: 0.8335
Epoch 19/150
142/142 [==============================] - 8s 58ms/step - loss: 0.1446 - accuracy: 0.9550 - f1_m: 0.9503 - val_loss: 1.1137 - val_accuracy: 0.7030 - val_f1_m: 0.7101
Epoch 20/150
142/142 [==============================] - 8s 58ms/step - loss: 0.1364 - accuracy: 0.9589 - f1_m: 0.9515 - val_loss: 0.4181 - val_accuracy: 0.8987 - val_f1_m: 0.8968
Epoch 21/150
142/142 [==============================] - 8s 58ms/step - loss: 0.1271 - accuracy: 0.9672 - f1_m: 0.9670 - val_loss: 1.3989 - val_accuracy: 0.6323 - val_f1_m: 0.6348
Epoch 22/150
142/142 [==============================] - 8s 58ms/step - loss: 0.1189 - accuracy: 0.9791 - f1_m: 0.9761 - val_loss: 0.2421 - val_accuracy: 0.9627 - val_f1_m: 0.9621
Epoch 23/150
142/142 [==============================] - 8s 58ms/step - loss: 0.1137 - accuracy: 0.9725 - f1_m: 0.9673 - val_loss: 0.3460 - val_accuracy: 0.9243 - val_f1_m: 0.9271
Epoch 24/150
142/142 [==============================] - 9s 63ms/step - loss: 0.1077 - accuracy: 0.9761 - f1_m: 0.9766 - val_loss: 0.6000 - val_accuracy: 0.8423 - val_f1_m: 0.8513
Epoch 25/150
142/142 [==============================] - 9s 61ms/step - loss: 0.1008 - accuracy: 0.9849 - f1_m: 0.9823 - val_loss: 1.1650 - val_accuracy: 0.7193 - val_f1_m: 0.7236
Epoch 26/150
142/142 [==============================] - 9s 60ms/step - loss: 0.0961 - accuracy: 0.9802 - f1_m: 0.9766 - val_loss: 0.2894 - val_accuracy: 0.9407 - val_f1_m: 0.9414
Epoch 27/150
142/142 [==============================] - 9s 60ms/step - loss: 0.0916 - accuracy: 0.9788 - f1_m: 0.9794 - val_loss: 0.2154 - val_accuracy: 0.9623 - val_f1_m: 0.9620
Epoch 28/150
142/142 [==============================] - 9s 60ms/step - loss: 0.0864 - accuracy: 0.9833 - f1_m: 0.9828 - val_loss: 0.3559 - val_accuracy: 0.9137 - val_f1_m: 0.9186
Epoch 29/150
142/142 [==============================] - 9s 60ms/step - loss: 0.0821 - accuracy: 0.9842 - f1_m: 0.9836 - val_loss: 0.3152 - val_accuracy: 0.9240 - val_f1_m: 0.9246
Epoch 30/150
142/142 [==============================] - 9s 60ms/step - loss: 0.0787 - accuracy: 0.9786 - f1_m: 0.9756 - val_loss: 0.4253 - val_accuracy: 0.8940 - val_f1_m: 0.8963
Epoch 31/150
142/142 [==============================] - 9s 60ms/step - loss: 0.0777 - accuracy: 0.9708 - f1_m: 0.9683 - val_loss: 0.3910 - val_accuracy: 0.8980 - val_f1_m: 0.9015
Epoch 32/150
142/142 [==============================] - 9s 60ms/step - loss: 0.0751 - accuracy: 0.9640 - f1_m: 0.9575 - val_loss: 0.2983 - val_accuracy: 0.9233 - val_f1_m: 0.9237
Epoch 33/150
142/142 [==============================] - 9s 60ms/step - loss: 0.0696 - accuracy: 0.9743 - f1_m: 0.9740 - val_loss: 0.7200 - val_accuracy: 0.8033 - val_f1_m: 0.8049
Epoch 34/150
142/142 [==============================] - 9s 60ms/step - loss: 0.0635 - accuracy: 0.9880 - f1_m: 0.9848 - val_loss: 0.4449 - val_accuracy: 0.8853 - val_f1_m: 0.8876
Epoch 35/150
142/142 [==============================] - 9s 60ms/step - loss: 0.0720 - accuracy: 0.9328 - f1_m: 0.9323 - val_loss: 0.3653 - val_accuracy: 0.8993 - val_f1_m: 0.9022
Epoch 36/150
142/142 [==============================] - 9s 60ms/step - loss: 0.0591 - accuracy: 0.9791 - f1_m: 0.9785 - val_loss: 0.2860 - val_accuracy: 0.9253 - val_f1_m: 0.9257
Epoch 37/150
142/142 [==============================] - 9s 60ms/step - loss: 0.0553 - accuracy: 0.9850 - f1_m: 0.9827 - val_loss: 0.2114 - val_accuracy: 0.9507 - val_f1_m: 0.9528
Epoch 38/150
142/142 [==============================] - 9s 60ms/step - loss: 0.0538 - accuracy: 0.9793 - f1_m: 0.9799 - val_loss: 0.3763 - val_accuracy: 0.8900 - val_f1_m: 0.8921
Epoch 39/150
142/142 [==============================] - 9s 60ms/step - loss: 0.0496 - accuracy: 0.9885 - f1_m: 0.9886 - val_loss: 0.1814 - val_accuracy: 0.9607 - val_f1_m: 0.9608
Epoch 40/150
142/142 [==============================] - 9s 60ms/step - loss: 0.0474 - accuracy: 0.9886 - f1_m: 0.9875 - val_loss: 0.2953 - val_accuracy: 0.9200 - val_f1_m: 0.9246
Epoch 41/150
142/142 [==============================] - 9s 60ms/step - loss: 0.0446 - accuracy: 0.9907 - f1_m: 0.9903 - val_loss: 0.1519 - val_accuracy: 0.9680 - val_f1_m: 0.9691
Epoch 42/150
142/142 [==============================] - 9s 60ms/step - loss: 0.0424 - accuracy: 0.9914 - f1_m: 0.9894 - val_loss: 0.1442 - val_accuracy: 0.9700 - val_f1_m: 0.9718
Epoch 43/150
142/142 [==============================] - 9s 60ms/step - loss: 0.0405 - accuracy: 0.9898 - f1_m: 0.9883 - val_loss: 0.2474 - val_accuracy: 0.9360 - val_f1_m: 0.9341
Epoch 44/150
142/142 [==============================] - 9s 60ms/step - loss: 0.0388 - accuracy: 0.9890 - f1_m: 0.9890 - val_loss: 0.1216 - val_accuracy: 0.9737 - val_f1_m: 0.9740
Epoch 45/150
142/142 [==============================] - 9s 60ms/step - loss: 0.0365 - accuracy: 0.9938 - f1_m: 0.9939 - val_loss: 0.2129 - val_accuracy: 0.9447 - val_f1_m: 0.9440
Epoch 46/150
142/142 [==============================] - 9s 60ms/step - loss: 0.0345 - accuracy: 0.9929 - f1_m: 0.9930 - val_loss: 0.1238 - val_accuracy: 0.9753 - val_f1_m: 0.9756
Epoch 47/150
142/142 [==============================] - 9s 60ms/step - loss: 0.0329 - accuracy: 0.9942 - f1_m: 0.9938 - val_loss: 0.1584 - val_accuracy: 0.9627 - val_f1_m: 0.9603
Epoch 48/150
142/142 [==============================] - 9s 60ms/step - loss: 0.0318 - accuracy: 0.9912 - f1_m: 0.9890 - val_loss: 0.1449 - val_accuracy: 0.9683 - val_f1_m: 0.9673
Epoch 49/150
142/142 [==============================] - 9s 60ms/step - loss: 0.0305 - accuracy: 0.9919 - f1_m: 0.9920 - val_loss: 0.1167 - val_accuracy: 0.9767 - val_f1_m: 0.9754
Epoch 50/150
142/142 [==============================] - 9s 60ms/step - loss: 0.0286 - accuracy: 0.9928 - f1_m: 0.9915 - val_loss: 0.1702 - val_accuracy: 0.9553 - val_f1_m: 0.9573
Epoch 51/150
142/142 [==============================] - 9s 60ms/step - loss: 0.0277 - accuracy: 0.9911 - f1_m: 0.9908 - val_loss: 0.1081 - val_accuracy: 0.9783 - val_f1_m: 0.9789
Epoch 52/150
142/142 [==============================] - 9s 61ms/step - loss: 0.0260 - accuracy: 0.9945 - f1_m: 0.9910 - val_loss: 0.1234 - val_accuracy: 0.9700 - val_f1_m: 0.9707
Epoch 53/150
142/142 [==============================] - 9s 61ms/step - loss: 0.0256 - accuracy: 0.9893 - f1_m: 0.9889 - val_loss: 0.4191 - val_accuracy: 0.8710 - val_f1_m: 0.8741
Epoch 54/150
142/142 [==============================] - 9s 60ms/step - loss: 0.0237 - accuracy: 0.9940 - f1_m: 0.9939 - val_loss: 0.1225 - val_accuracy: 0.9720 - val_f1_m: 0.9712
Epoch 55/150
142/142 [==============================] - 9s 60ms/step - loss: 0.0230 - accuracy: 0.9928 - f1_m: 0.9915 - val_loss: 0.4859 - val_accuracy: 0.9007 - val_f1_m: 0.9036
Epoch 56/150
142/142 [==============================] - 9s 60ms/step - loss: 0.0263 - accuracy: 0.9726 - f1_m: 0.9709 - val_loss: 0.1645 - val_accuracy: 0.9607 - val_f1_m: 0.9605
Epoch 57/150
142/142 [==============================] - 9s 60ms/step - loss: 0.0215 - accuracy: 0.9915 - f1_m: 0.9898 - val_loss: 0.1049 - val_accuracy: 0.9753 - val_f1_m: 0.9751
Epoch 58/150
142/142 [==============================] - 9s 60ms/step - loss: 0.0204 - accuracy: 0.9935 - f1_m: 0.9903 - val_loss: 0.1424 - val_accuracy: 0.9623 - val_f1_m: 0.9631
Epoch 59/150
142/142 [==============================] - 9s 60ms/step - loss: 0.0228 - accuracy: 0.9781 - f1_m: 0.9758 - val_loss: 0.1697 - val_accuracy: 0.9570 - val_f1_m: 0.9571
Epoch 60/150
142/142 [==============================] - 9s 60ms/step - loss: 0.0226 - accuracy: 0.9797 - f1_m: 0.9805 - val_loss: 0.1579 - val_accuracy: 0.9550 - val_f1_m: 0.9555
Epoch 61/150
142/142 [==============================] - 9s 60ms/step - loss: 0.0185 - accuracy: 0.9930 - f1_m: 0.9920 - val_loss: 0.2244 - val_accuracy: 0.9367 - val_f1_m: 0.9342
Epoch 62/150
142/142 [==============================] - 9s 60ms/step - loss: 0.0197 - accuracy: 0.9814 - f1_m: 0.9795 - val_loss: 0.6162 - val_accuracy: 0.7960 - val_f1_m: 0.7995
Epoch 63/150
142/142 [==============================] - 9s 60ms/step - loss: 0.0175 - accuracy: 0.9910 - f1_m: 0.9906 - val_loss: 0.1252 - val_accuracy: 0.9673 - val_f1_m: 0.9685
Epoch 64/150
142/142 [==============================] - 9s 60ms/step - loss: 0.0160 - accuracy: 0.9950 - f1_m: 0.9949 - val_loss: 0.0943 - val_accuracy: 0.9743 - val_f1_m: 0.9757
Epoch 65/150
142/142 [==============================] - 9s 60ms/step - loss: 0.0155 - accuracy: 0.9940 - f1_m: 0.9916 - val_loss: 0.1663 - val_accuracy: 0.9533 - val_f1_m: 0.9550
Epoch 66/150
142/142 [==============================] - 9s 60ms/step - loss: 0.0156 - accuracy: 0.9900 - f1_m: 0.9891 - val_loss: 0.1173 - val_accuracy: 0.9667 - val_f1_m: 0.9681
Epoch 67/150
142/142 [==============================] - 9s 60ms/step - loss: 0.0150 - accuracy: 0.9896 - f1_m: 0.9884 - val_loss: 0.0849 - val_accuracy: 0.9793 - val_f1_m: 0.9791
Epoch 68/150
142/142 [==============================] - 9s 61ms/step - loss: 0.0140 - accuracy: 0.9930 - f1_m: 0.9918 - val_loss: 0.1158 - val_accuracy: 0.9683 - val_f1_m: 0.9691
Epoch 69/150
142/142 [==============================] - 9s 60ms/step - loss: 0.0135 - accuracy: 0.9925 - f1_m: 0.9929 - val_loss: 0.1156 - val_accuracy: 0.9660 - val_f1_m: 0.9659
Epoch 70/150
142/142 [==============================] - 9s 60ms/step - loss: 0.0133 - accuracy: 0.9926 - f1_m: 0.9894 - val_loss: 0.0879 - val_accuracy: 0.9787 - val_f1_m: 0.9779
Epoch 71/150
142/142 [==============================] - 9s 60ms/step - loss: 0.0136 - accuracy: 0.9888 - f1_m: 0.9869 - val_loss: 0.1888 - val_accuracy: 0.9450 - val_f1_m: 0.9449
Epoch 72/150
142/142 [==============================] - 9s 60ms/step - loss: 0.0122 - accuracy: 0.9934 - f1_m: 0.9910 - val_loss: 0.1258 - val_accuracy: 0.9663 - val_f1_m: 0.9661
Epoch 73/150
142/142 [==============================] - 9s 61ms/step - loss: 0.0127 - accuracy: 0.9869 - f1_m: 0.9842 - val_loss: 0.1719 - val_accuracy: 0.9477 - val_f1_m: 0.9494
Epoch 74/150
142/142 [==============================] - 9s 61ms/step - loss: 0.0138 - accuracy: 0.9817 - f1_m: 0.9809 - val_loss: 0.1942 - val_accuracy: 0.9450 - val_f1_m: 0.9452
Epoch 75/150
142/142 [==============================] - 9s 60ms/step - loss: 0.0111 - accuracy: 0.9944 - f1_m: 0.9939 - val_loss: 0.0703 - val_accuracy: 0.9817 - val_f1_m: 0.9823
Epoch 76/150
142/142 [==============================] - 9s 61ms/step - loss: 0.0105 - accuracy: 0.9955 - f1_m: 0.9953 - val_loss: 0.1181 - val_accuracy: 0.9643 - val_f1_m: 0.9675
Epoch 77/150
142/142 [==============================] - 9s 61ms/step - loss: 0.0100 - accuracy: 0.9968 - f1_m: 0.9971 - val_loss: 0.0798 - val_accuracy: 0.9790 - val_f1_m: 0.9792
Epoch 78/150
142/142 [==============================] - 9s 60ms/step - loss: 0.0097 - accuracy: 0.9959 - f1_m: 0.9962 - val_loss: 0.0728 - val_accuracy: 0.9830 - val_f1_m: 0.9829
Epoch 79/150
142/142 [==============================] - 9s 60ms/step - loss: 0.0092 - accuracy: 0.9977 - f1_m: 0.9977 - val_loss: 0.0903 - val_accuracy: 0.9793 - val_f1_m: 0.9793
Epoch 80/150
142/142 [==============================] - 9s 60ms/step - loss: 0.0088 - accuracy: 0.9971 - f1_m: 0.9968 - val_loss: 0.0851 - val_accuracy: 0.9770 - val_f1_m: 0.9776
Epoch 81/150
142/142 [==============================] - 9s 60ms/step - loss: 0.0086 - accuracy: 0.9972 - f1_m: 0.9968 - val_loss: 0.0781 - val_accuracy: 0.9817 - val_f1_m: 0.9818
Epoch 82/150
142/142 [==============================] - 9s 60ms/step - loss: 0.0090 - accuracy: 0.9942 - f1_m: 0.9942 - val_loss: 0.1855 - val_accuracy: 0.9407 - val_f1_m: 0.9417
Epoch 83/150
142/142 [==============================] - 9s 61ms/step - loss: 0.0081 - accuracy: 0.9970 - f1_m: 0.9953 - val_loss: 0.1026 - val_accuracy: 0.9723 - val_f1_m: 0.9716
Epoch 84/150
142/142 [==============================] - 9s 60ms/step - loss: 0.0088 - accuracy: 0.9926 - f1_m: 0.9906 - val_loss: 0.0741 - val_accuracy: 0.9820 - val_f1_m: 0.9826
Epoch 85/150
142/142 [==============================] - 9s 60ms/step - loss: 0.0086 - accuracy: 0.9942 - f1_m: 0.9940 - val_loss: 0.1409 - val_accuracy: 0.9627 - val_f1_m: 0.9612
Epoch 85: early stopping
CPU times: total: 4min 59s
Wall time: 12min 12s

<keras.callbacks.History at 0x1de5cba0730>

%%time
CustomModel_31_DA = create_Custom_model((31, 31, 1), 15)
CustomModel_31_DA.compile(optimizer=SGD(learning_rate=learning_rate, momentum=momentum), loss='categorical_crossentropy', metrics=['accuracy',f1_m])
CustomModel_31_DA.fit(train_generator_31, epochs=150, validation_data=dataset_test_31, class_weight=class_weights_dict,callbacks=[EarlyStopping(monitor='val_loss', patience=10, verbose=1, mode='min')],steps_per_epoch=steps_per_epoch)

Epoch 1/150
142/142 [==============================] - 5s 27ms/step - loss: 0.4665 - accuracy: 0.3095 - f1_m: 0.1313 - val_loss: 2.8437 - val_accuracy: 0.1683 - val_f1_m: 0.0026
Epoch 2/150
142/142 [==============================] - 4s 24ms/step - loss: 0.4033 - accuracy: 0.5836 - f1_m: 0.4956 - val_loss: 2.1785 - val_accuracy: 0.3510 - val_f1_m: 0.2680
Epoch 3/150
142/142 [==============================] - 4s 24ms/step - loss: 0.3642 - accuracy: 0.7113 - f1_m: 0.6815 - val_loss: 1.3861 - val_accuracy: 0.6620 - val_f1_m: 0.6248
Epoch 4/150
142/142 [==============================] - 4s 24ms/step - loss: 0.3348 - accuracy: 0.7816 - f1_m: 0.7702 - val_loss: 1.1378 - val_accuracy: 0.7320 - val_f1_m: 0.7182
Epoch 5/150
142/142 [==============================] - 4s 25ms/step - loss: 0.3112 - accuracy: 0.8322 - f1_m: 0.8230 - val_loss: 0.7999 - val_accuracy: 0.8383 - val_f1_m: 0.8354
Epoch 6/150
142/142 [==============================] - 4s 25ms/step - loss: 0.2903 - accuracy: 0.8662 - f1_m: 0.8547 - val_loss: 2.0145 - val_accuracy: 0.5390 - val_f1_m: 0.5296
Epoch 7/150
142/142 [==============================] - 4s 24ms/step - loss: 0.2782 - accuracy: 0.8504 - f1_m: 0.8438 - val_loss: 1.0195 - val_accuracy: 0.7480 - val_f1_m: 0.7458
Epoch 8/150
142/142 [==============================] - 4s 24ms/step - loss: 0.2579 - accuracy: 0.8943 - f1_m: 0.8912 - val_loss: 1.4138 - val_accuracy: 0.6440 - val_f1_m: 0.6509
Epoch 9/150
142/142 [==============================] - 3s 24ms/step - loss: 0.2455 - accuracy: 0.8949 - f1_m: 0.8923 - val_loss: 0.6127 - val_accuracy: 0.8830 - val_f1_m: 0.8764
Epoch 10/150
142/142 [==============================] - 3s 24ms/step - loss: 0.2284 - accuracy: 0.9267 - f1_m: 0.9240 - val_loss: 0.5663 - val_accuracy: 0.8873 - val_f1_m: 0.8915
Epoch 11/150
142/142 [==============================] - 3s 24ms/step - loss: 0.2153 - accuracy: 0.9366 - f1_m: 0.9333 - val_loss: 0.8124 - val_accuracy: 0.7903 - val_f1_m: 0.7881
Epoch 12/150
142/142 [==============================] - 3s 24ms/step - loss: 0.2072 - accuracy: 0.9196 - f1_m: 0.9183 - val_loss: 0.6137 - val_accuracy: 0.8697 - val_f1_m: 0.8718
Epoch 13/150
142/142 [==============================] - 3s 24ms/step - loss: 0.1923 - accuracy: 0.9465 - f1_m: 0.9423 - val_loss: 0.5988 - val_accuracy: 0.8677 - val_f1_m: 0.8670
Epoch 14/150
142/142 [==============================] - 4s 24ms/step - loss: 0.1819 - accuracy: 0.9523 - f1_m: 0.9490 - val_loss: 0.8337 - val_accuracy: 0.7867 - val_f1_m: 0.7888
Epoch 15/150
142/142 [==============================] - 4s 24ms/step - loss: 0.1765 - accuracy: 0.9344 - f1_m: 0.9320 - val_loss: 0.5611 - val_accuracy: 0.8763 - val_f1_m: 0.8810
Epoch 16/150
142/142 [==============================] - 4s 24ms/step - loss: 0.1628 - accuracy: 0.9593 - f1_m: 0.9587 - val_loss: 0.5524 - val_accuracy: 0.8757 - val_f1_m: 0.8756
Epoch 17/150
142/142 [==============================] - 3s 24ms/step - loss: 0.1540 - accuracy: 0.9637 - f1_m: 0.9612 - val_loss: 0.6562 - val_accuracy: 0.8427 - val_f1_m: 0.8440
Epoch 18/150
142/142 [==============================] - 3s 24ms/step - loss: 0.1515 - accuracy: 0.9337 - f1_m: 0.9342 - val_loss: 0.6055 - val_accuracy: 0.8523 - val_f1_m: 0.8572
Epoch 19/150
142/142 [==============================] - 4s 25ms/step - loss: 0.1390 - accuracy: 0.9639 - f1_m: 0.9638 - val_loss: 0.5870 - val_accuracy: 0.8553 - val_f1_m: 0.8569
Epoch 20/150
142/142 [==============================] - 4s 25ms/step - loss: 0.1320 - accuracy: 0.9623 - f1_m: 0.9604 - val_loss: 0.6023 - val_accuracy: 0.8483 - val_f1_m: 0.8484
Epoch 21/150
142/142 [==============================] - 4s 25ms/step - loss: 0.1259 - accuracy: 0.9626 - f1_m: 0.9634 - val_loss: 0.3880 - val_accuracy: 0.9167 - val_f1_m: 0.9180
Epoch 22/150
142/142 [==============================] - 4s 25ms/step - loss: 0.1176 - accuracy: 0.9752 - f1_m: 0.9751 - val_loss: 0.3770 - val_accuracy: 0.9150 - val_f1_m: 0.9144
Epoch 23/150
142/142 [==============================] - 4s 24ms/step - loss: 0.1116 - accuracy: 0.9718 - f1_m: 0.9707 - val_loss: 0.6312 - val_accuracy: 0.8497 - val_f1_m: 0.8545
Epoch 24/150
142/142 [==============================] - 4s 24ms/step - loss: 0.1090 - accuracy: 0.9572 - f1_m: 0.9549 - val_loss: 0.5307 - val_accuracy: 0.8643 - val_f1_m: 0.8651
Epoch 25/150
142/142 [==============================] - 4s 24ms/step - loss: 0.1041 - accuracy: 0.9581 - f1_m: 0.9583 - val_loss: 0.3986 - val_accuracy: 0.9067 - val_f1_m: 0.9125
Epoch 26/150
142/142 [==============================] - 4s 25ms/step - loss: 0.0954 - accuracy: 0.9794 - f1_m: 0.9781 - val_loss: 0.4389 - val_accuracy: 0.8917 - val_f1_m: 0.8939
Epoch 27/150
142/142 [==============================] - 3s 24ms/step - loss: 0.0909 - accuracy: 0.9753 - f1_m: 0.9732 - val_loss: 0.3898 - val_accuracy: 0.9030 - val_f1_m: 0.9031
Epoch 28/150
142/142 [==============================] - 4s 24ms/step - loss: 0.0863 - accuracy: 0.9783 - f1_m: 0.9765 - val_loss: 0.8918 - val_accuracy: 0.7550 - val_f1_m: 0.7546
Epoch 29/150
142/142 [==============================] - 4s 25ms/step - loss: 0.0869 - accuracy: 0.9557 - f1_m: 0.9538 - val_loss: 0.3368 - val_accuracy: 0.9240 - val_f1_m: 0.9224
Epoch 30/150
142/142 [==============================] - 3s 24ms/step - loss: 0.0796 - accuracy: 0.9714 - f1_m: 0.9706 - val_loss: 0.3471 - val_accuracy: 0.9110 - val_f1_m: 0.9146
Epoch 31/150
142/142 [==============================] - 3s 24ms/step - loss: 0.0739 - accuracy: 0.9822 - f1_m: 0.9822 - val_loss: 0.3067 - val_accuracy: 0.9247 - val_f1_m: 0.9274
Epoch 32/150
142/142 [==============================] - 4s 24ms/step - loss: 0.0701 - accuracy: 0.9833 - f1_m: 0.9811 - val_loss: 0.3183 - val_accuracy: 0.9283 - val_f1_m: 0.9247
Epoch 33/150
142/142 [==============================] - 4s 24ms/step - loss: 0.0674 - accuracy: 0.9801 - f1_m: 0.9796 - val_loss: 0.2532 - val_accuracy: 0.9443 - val_f1_m: 0.9466
Epoch 34/150
142/142 [==============================] - 4s 24ms/step - loss: 0.0627 - accuracy: 0.9878 - f1_m: 0.9872 - val_loss: 0.2529 - val_accuracy: 0.9460 - val_f1_m: 0.9479
Epoch 35/150
142/142 [==============================] - 4s 25ms/step - loss: 0.0600 - accuracy: 0.9870 - f1_m: 0.9858 - val_loss: 0.2572 - val_accuracy: 0.9437 - val_f1_m: 0.9439
Epoch 36/150
142/142 [==============================] - 4s 25ms/step - loss: 0.0574 - accuracy: 0.9853 - f1_m: 0.9845 - val_loss: 0.3107 - val_accuracy: 0.9243 - val_f1_m: 0.9236
Epoch 37/150
142/142 [==============================] - 4s 24ms/step - loss: 0.0546 - accuracy: 0.9856 - f1_m: 0.9854 - val_loss: 0.3073 - val_accuracy: 0.9200 - val_f1_m: 0.9210
Epoch 38/150
142/142 [==============================] - 4s 24ms/step - loss: 0.0516 - accuracy: 0.9867 - f1_m: 0.9866 - val_loss: 0.2543 - val_accuracy: 0.9397 - val_f1_m: 0.9377
Epoch 39/150
142/142 [==============================] - 4s 25ms/step - loss: 0.0490 - accuracy: 0.9887 - f1_m: 0.9887 - val_loss: 0.3247 - val_accuracy: 0.9173 - val_f1_m: 0.9188
Epoch 40/150
142/142 [==============================] - 4s 25ms/step - loss: 0.0465 - accuracy: 0.9896 - f1_m: 0.9898 - val_loss: 0.2430 - val_accuracy: 0.9427 - val_f1_m: 0.9424
Epoch 41/150
142/142 [==============================] - 3s 24ms/step - loss: 0.0444 - accuracy: 0.9911 - f1_m: 0.9881 - val_loss: 0.2307 - val_accuracy: 0.9400 - val_f1_m: 0.9405
Epoch 42/150
142/142 [==============================] - 3s 24ms/step - loss: 0.0435 - accuracy: 0.9838 - f1_m: 0.9840 - val_loss: 0.2198 - val_accuracy: 0.9473 - val_f1_m: 0.9464
Epoch 43/150
142/142 [==============================] - 4s 25ms/step - loss: 0.0405 - accuracy: 0.9901 - f1_m: 0.9873 - val_loss: 0.2339 - val_accuracy: 0.9413 - val_f1_m: 0.9411
Epoch 44/150
142/142 [==============================] - 4s 25ms/step - loss: 0.0392 - accuracy: 0.9865 - f1_m: 0.9859 - val_loss: 0.2213 - val_accuracy: 0.9473 - val_f1_m: 0.9471
Epoch 45/150
142/142 [==============================] - 4s 25ms/step - loss: 0.0369 - accuracy: 0.9904 - f1_m: 0.9904 - val_loss: 0.2261 - val_accuracy: 0.9423 - val_f1_m: 0.9418
Epoch 46/150
142/142 [==============================] - 4s 25ms/step - loss: 0.0355 - accuracy: 0.9916 - f1_m: 0.9854 - val_loss: 0.3039 - val_accuracy: 0.9183 - val_f1_m: 0.9199
Epoch 47/150
142/142 [==============================] - 4s 25ms/step - loss: 0.0371 - accuracy: 0.9715 - f1_m: 0.9706 - val_loss: 0.2578 - val_accuracy: 0.9363 - val_f1_m: 0.9361
Epoch 48/150
142/142 [==============================] - 4s 25ms/step - loss: 0.0324 - accuracy: 0.9910 - f1_m: 0.9892 - val_loss: 0.2120 - val_accuracy: 0.9493 - val_f1_m: 0.9504
Epoch 49/150
142/142 [==============================] - 4s 25ms/step - loss: 0.0320 - accuracy: 0.9828 - f1_m: 0.9818 - val_loss: 0.2591 - val_accuracy: 0.9303 - val_f1_m: 0.9318
Epoch 50/150
142/142 [==============================] - 4s 25ms/step - loss: 0.0303 - accuracy: 0.9862 - f1_m: 0.9859 - val_loss: 0.1995 - val_accuracy: 0.9473 - val_f1_m: 0.9473
Epoch 51/150
142/142 [==============================] - 4s 25ms/step - loss: 0.0281 - accuracy: 0.9926 - f1_m: 0.9902 - val_loss: 0.2532 - val_accuracy: 0.9257 - val_f1_m: 0.9282
Epoch 52/150
142/142 [==============================] - 4s 25ms/step - loss: 0.0275 - accuracy: 0.9901 - f1_m: 0.9891 - val_loss: 0.2212 - val_accuracy: 0.9413 - val_f1_m: 0.9419
Epoch 53/150
142/142 [==============================] - 4s 26ms/step - loss: 0.0257 - accuracy: 0.9930 - f1_m: 0.9928 - val_loss: 0.2817 - val_accuracy: 0.9240 - val_f1_m: 0.9221
Epoch 54/150
142/142 [==============================] - 4s 25ms/step - loss: 0.0250 - accuracy: 0.9914 - f1_m: 0.9894 - val_loss: 0.2427 - val_accuracy: 0.9320 - val_f1_m: 0.9338
Epoch 55/150
142/142 [==============================] - 4s 25ms/step - loss: 0.0279 - accuracy: 0.9730 - f1_m: 0.9743 - val_loss: 0.2573 - val_accuracy: 0.9243 - val_f1_m: 0.9271
Epoch 56/150
142/142 [==============================] - 4s 25ms/step - loss: 0.0248 - accuracy: 0.9847 - f1_m: 0.9826 - val_loss: 0.2039 - val_accuracy: 0.9443 - val_f1_m: 0.9478
Epoch 57/150
142/142 [==============================] - 4s 25ms/step - loss: 0.0230 - accuracy: 0.9869 - f1_m: 0.9866 - val_loss: 0.1969 - val_accuracy: 0.9490 - val_f1_m: 0.9529
Epoch 58/150
142/142 [==============================] - 4s 25ms/step - loss: 0.0209 - accuracy: 0.9939 - f1_m: 0.9929 - val_loss: 0.2929 - val_accuracy: 0.9163 - val_f1_m: 0.9165
Epoch 59/150
142/142 [==============================] - 4s 24ms/step - loss: 0.0203 - accuracy: 0.9932 - f1_m: 0.9928 - val_loss: 0.2001 - val_accuracy: 0.9450 - val_f1_m: 0.9459
Epoch 60/150
142/142 [==============================] - 3s 24ms/step - loss: 0.0192 - accuracy: 0.9947 - f1_m: 0.9941 - val_loss: 0.1641 - val_accuracy: 0.9590 - val_f1_m: 0.9600
Epoch 61/150
142/142 [==============================] - 4s 24ms/step - loss: 0.0185 - accuracy: 0.9945 - f1_m: 0.9944 - val_loss: 0.1785 - val_accuracy: 0.9527 - val_f1_m: 0.9518
Epoch 62/150
142/142 [==============================] - 4s 25ms/step - loss: 0.0176 - accuracy: 0.9952 - f1_m: 0.9950 - val_loss: 0.1783 - val_accuracy: 0.9520 - val_f1_m: 0.9513
Epoch 63/150
142/142 [==============================] - 4s 25ms/step - loss: 0.0172 - accuracy: 0.9930 - f1_m: 0.9933 - val_loss: 0.1816 - val_accuracy: 0.9540 - val_f1_m: 0.9530
Epoch 64/150
142/142 [==============================] - 4s 25ms/step - loss: 0.0162 - accuracy: 0.9959 - f1_m: 0.9953 - val_loss: 0.1750 - val_accuracy: 0.9507 - val_f1_m: 0.9533
Epoch 65/150
142/142 [==============================] - 4s 24ms/step - loss: 0.0160 - accuracy: 0.9948 - f1_m: 0.9933 - val_loss: 0.2057 - val_accuracy: 0.9467 - val_f1_m: 0.9488
Epoch 66/150
142/142 [==============================] - 4s 24ms/step - loss: 0.0159 - accuracy: 0.9905 - f1_m: 0.9910 - val_loss: 0.1829 - val_accuracy: 0.9520 - val_f1_m: 0.9525
Epoch 67/150
142/142 [==============================] - 4s 24ms/step - loss: 0.0155 - accuracy: 0.9898 - f1_m: 0.9891 - val_loss: 0.3138 - val_accuracy: 0.9083 - val_f1_m: 0.9092
Epoch 68/150
142/142 [==============================] - 3s 24ms/step - loss: 0.0177 - accuracy: 0.9750 - f1_m: 0.9740 - val_loss: 0.2753 - val_accuracy: 0.9207 - val_f1_m: 0.9227
Epoch 69/150
142/142 [==============================] - 3s 24ms/step - loss: 0.0157 - accuracy: 0.9878 - f1_m: 0.9866 - val_loss: 0.2046 - val_accuracy: 0.9390 - val_f1_m: 0.9413
Epoch 70/150
142/142 [==============================] - 3s 24ms/step - loss: 0.0144 - accuracy: 0.9903 - f1_m: 0.9868 - val_loss: 0.2280 - val_accuracy: 0.9383 - val_f1_m: 0.9393
Epoch 70: early stopping
CPU times: total: 1min 27s
Wall time: 4min 13s

<keras.callbacks.History at 0x1de2fa0bf10>

evaluate_and_save_model(CustomModel_128_DA, 'CustomModel_128_DA', 64,save_models=False)
evaluate_and_save_model(CustomModel_31_DA, 'CustomModel_31_DA', 64,save_models=False)

Model Name            CustomModel_128_DA
Epochs                                85
Batch Size                            64
Train Loss                      0.009669
Val Loss                        0.072805
Train Acc                       0.995902
Val Acc                            0.983
[Train - Val] Acc               0.012902
[Train - Val] Loss             -0.063136
Name: CustomModel_128_DA, dtype: object
Model Name            CustomModel_31_DA
Epochs                               70
Batch Size                           64
Train Loss                      0.01923
Val Loss                       0.164061
Train Acc                      0.994683
Val Acc                           0.959
[Train - Val] Acc              0.035683
[Train - Val] Loss             -0.14483
Name: CustomModel_31_DA, dtype: object

customModel_history_128_DA = json.load(open('./models/CustomModel_128_DA_history.json', 'r'))
customModel_history_31_DA = json.load(open('./models/CustomModel_31_DA_history.json', 'r'))
compare_curves(customModel_history_31_DA, customModel_history_128_DA, '31x31', '128x128')

Observations:

• We can see that with both data augmented images and a dropout layer, our accuracy and loss graph has increased amounts of flucutations
  • This can be becauses Data Augmentation includes more variablilty in our dataset by applying random transformations to the training images
• With the inclusion of Data augmentation, we have effectively prevented overfitting as compared to our custom model trained on non agumented images.

Custom Resnet Network

Resnets (Residual Networks) were first introduced in 2015 ,by Microsoft researchers, Kaiming He, Xiangyu Zhang, Shaoqing Ren and Jian Sun. In their paper "Deep Residual Learning for Image Recognition" [4]. They talked on the problem of how deeper neural networks are harder to train and they provide evidence on how Resnets are easier to optimize and learn in much deeper neural networks as compared to other neural networks such as VGG nets during 2015.

First of all why Resnet?

Resnet's unqiue architechture of skipping connections backwards from later layers to intital filters, where there are 2 pathways for the gradient to transit back to the inputlayer while tranversing a residual block. allowing us to combat a big problem in neural networks, vanishing/exploding gradients. Where when our model network gets too deep and the gradient calculated from backpropagation gradually goes to 0 after multiple applications of the chain rule. Resnet allows us to have deeper and more complex models without encountering the problem of vanishing and exploding gradients.

Architechture of a regular residual network

Source

Here for example is a 34 layer Resnet, where the main thing that stands out is the skip connections it has. While th architecture is organized into four stages with 3, 4, 6, and 3 residual blocks.The residual blocks consist of 2 stacked convolutional layers with batch normalization and ReLU activations. After the last stage, a global average pooling layer is applied to reduce spatial dimensions. We can see that there are 2 different types of skip connections in this model (dotted and non-dotted lines). An identity skip connection and a projection skip connection. What is the difference between the 2? In projection skip connections,it is similair to a normal identity skip connection except additional operations(eg. a 1x1 convolutional layer) are applied to the input to match the spatial dimensions of the output of our residual block.

resnet50_example = tf.keras.applications.ResNet50(
    include_top=False,
    weights=None,
    input_shape=(128, 128, 1),
    pooling='avg',
    classes=15,
)
resnet50_example.summary()

Model: "resnet50"
__________________________________________________________________________________________________
 Layer (type)                   Output Shape         Param #     Connected to                     
==================================================================================================
 input_2 (InputLayer)           [(None, 128, 128, 1  0           []                               
                                )]                                                                
                                                                                                  
 conv1_pad (ZeroPadding2D)      (None, 134, 134, 1)  0           ['input_2[0][0]']                
                                                                                                  
 conv1_conv (Conv2D)            (None, 64, 64, 64)   3200        ['conv1_pad[0][0]']              
                                                                                                  
 conv1_bn (BatchNormalization)  (None, 64, 64, 64)   256         ['conv1_conv[0][0]']             
                                                                                                  
 conv1_relu (Activation)        (None, 64, 64, 64)   0           ['conv1_bn[0][0]']               
                                                                                                  
 pool1_pad (ZeroPadding2D)      (None, 66, 66, 64)   0           ['conv1_relu[0][0]']             
                                                                                                  
 pool1_pool (MaxPooling2D)      (None, 32, 32, 64)   0           ['pool1_pad[0][0]']              
                                                                                                  
 conv2_block1_1_conv (Conv2D)   (None, 32, 32, 64)   4160        ['pool1_pool[0][0]']             
                                                                                                  
 conv2_block1_1_bn (BatchNormal  (None, 32, 32, 64)  256         ['conv2_block1_1_conv[0][0]']    
 ization)                                                                                         
                                                                                                  
 conv2_block1_1_relu (Activatio  (None, 32, 32, 64)  0           ['conv2_block1_1_bn[0][0]']      
 n)                                                                                               
                                                                                                  
 conv2_block1_2_conv (Conv2D)   (None, 32, 32, 64)   36928       ['conv2_block1_1_relu[0][0]']    
                                                                                                  
 conv2_block1_2_bn (BatchNormal  (None, 32, 32, 64)  256         ['conv2_block1_2_conv[0][0]']    
 ization)                                                                                         
                                                                                                  
 conv2_block1_2_relu (Activatio  (None, 32, 32, 64)  0           ['conv2_block1_2_bn[0][0]']      
 n)                                                                                               
                                                                                                  
 conv2_block1_0_conv (Conv2D)   (None, 32, 32, 256)  16640       ['pool1_pool[0][0]']             
                                                                                                  
 conv2_block1_3_conv (Conv2D)   (None, 32, 32, 256)  16640       ['conv2_block1_2_relu[0][0]']    
                                                                                                  
 conv2_block1_0_bn (BatchNormal  (None, 32, 32, 256)  1024       ['conv2_block1_0_conv[0][0]']    
 ization)                                                                                         
                                                                                                  
 conv2_block1_3_bn (BatchNormal  (None, 32, 32, 256)  1024       ['conv2_block1_3_conv[0][0]']    
 ization)                                                                                         
                                                                                                  
 conv2_block1_add (Add)         (None, 32, 32, 256)  0           ['conv2_block1_0_bn[0][0]',      
                                                                  'conv2_block1_3_bn[0][0]']      
                                                                                                  
 conv2_block1_out (Activation)  (None, 32, 32, 256)  0           ['conv2_block1_add[0][0]']       
                                                                                                  
 conv2_block2_1_conv (Conv2D)   (None, 32, 32, 64)   16448       ['conv2_block1_out[0][0]']       
                                                                                                  
 conv2_block2_1_bn (BatchNormal  (None, 32, 32, 64)  256         ['conv2_block2_1_conv[0][0]']    
 ization)                                                                                         
                                                                                                  
 conv2_block2_1_relu (Activatio  (None, 32, 32, 64)  0           ['conv2_block2_1_bn[0][0]']      
 n)                                                                                               
                                                                                                  
 conv2_block2_2_conv (Conv2D)   (None, 32, 32, 64)   36928       ['conv2_block2_1_relu[0][0]']    
                                                                                                  
 conv2_block2_2_bn (BatchNormal  (None, 32, 32, 64)  256         ['conv2_block2_2_conv[0][0]']    
 ization)                                                                                         
                                                                                                  
 conv2_block2_2_relu (Activatio  (None, 32, 32, 64)  0           ['conv2_block2_2_bn[0][0]']      
 n)                                                                                               
                                                                                                  
 conv2_block2_3_conv (Conv2D)   (None, 32, 32, 256)  16640       ['conv2_block2_2_relu[0][0]']    
                                                                                                  
 conv2_block2_3_bn (BatchNormal  (None, 32, 32, 256)  1024       ['conv2_block2_3_conv[0][0]']    
 ization)                                                                                         
                                                                                                  
 conv2_block2_add (Add)         (None, 32, 32, 256)  0           ['conv2_block1_out[0][0]',       
                                                                  'conv2_block2_3_bn[0][0]']      
                                                                                                  
 conv2_block2_out (Activation)  (None, 32, 32, 256)  0           ['conv2_block2_add[0][0]']       
                                                                                                  
 conv2_block3_1_conv (Conv2D)   (None, 32, 32, 64)   16448       ['conv2_block2_out[0][0]']       
                                                                                                  
 conv2_block3_1_bn (BatchNormal  (None, 32, 32, 64)  256         ['conv2_block3_1_conv[0][0]']    
 ization)                                                                                         
                                                                                                  
 conv2_block3_1_relu (Activatio  (None, 32, 32, 64)  0           ['conv2_block3_1_bn[0][0]']      
 n)                                                                                               
                                                                                                  
 conv2_block3_2_conv (Conv2D)   (None, 32, 32, 64)   36928       ['conv2_block3_1_relu[0][0]']    
                                                                                                  
 conv2_block3_2_bn (BatchNormal  (None, 32, 32, 64)  256         ['conv2_block3_2_conv[0][0]']    
 ization)                                                                                         
                                                                                                  
 conv2_block3_2_relu (Activatio  (None, 32, 32, 64)  0           ['conv2_block3_2_bn[0][0]']      
 n)                                                                                               
                                                                                                  
 conv2_block3_3_conv (Conv2D)   (None, 32, 32, 256)  16640       ['conv2_block3_2_relu[0][0]']    
                                                                                                  
 conv2_block3_3_bn (BatchNormal  (None, 32, 32, 256)  1024       ['conv2_block3_3_conv[0][0]']    
 ization)                                                                                         
                                                                                                  
 conv2_block3_add (Add)         (None, 32, 32, 256)  0           ['conv2_block2_out[0][0]',       
                                                                  'conv2_block3_3_bn[0][0]']      
                                                                                                  
 conv2_block3_out (Activation)  (None, 32, 32, 256)  0           ['conv2_block3_add[0][0]']       
                                                                                                  
 conv3_block1_1_conv (Conv2D)   (None, 16, 16, 128)  32896       ['conv2_block3_out[0][0]']       
                                                                                                  
 conv3_block1_1_bn (BatchNormal  (None, 16, 16, 128)  512        ['conv3_block1_1_conv[0][0]']    
 ization)                                                                                         
                                                                                                  
 conv3_block1_1_relu (Activatio  (None, 16, 16, 128)  0          ['conv3_block1_1_bn[0][0]']      
 n)                                                                                               
                                                                                                  
 conv3_block1_2_conv (Conv2D)   (None, 16, 16, 128)  147584      ['conv3_block1_1_relu[0][0]']    
                                                                                                  
 conv3_block1_2_bn (BatchNormal  (None, 16, 16, 128)  512        ['conv3_block1_2_conv[0][0]']    
 ization)                                                                                         
                                                                                                  
 conv3_block1_2_relu (Activatio  (None, 16, 16, 128)  0          ['conv3_block1_2_bn[0][0]']      
 n)                                                                                               
                                                                                                  
 conv3_block1_0_conv (Conv2D)   (None, 16, 16, 512)  131584      ['conv2_block3_out[0][0]']       
                                                                                                  
 conv3_block1_3_conv (Conv2D)   (None, 16, 16, 512)  66048       ['conv3_block1_2_relu[0][0]']    
                                                                                                  
 conv3_block1_0_bn (BatchNormal  (None, 16, 16, 512)  2048       ['conv3_block1_0_conv[0][0]']    
 ization)                                                                                         
                                                                                                  
 conv3_block1_3_bn (BatchNormal  (None, 16, 16, 512)  2048       ['conv3_block1_3_conv[0][0]']    
 ization)                                                                                         
                                                                                                  
 conv3_block1_add (Add)         (None, 16, 16, 512)  0           ['conv3_block1_0_bn[0][0]',      
                                                                  'conv3_block1_3_bn[0][0]']      
                                                                                                  
 conv3_block1_out (Activation)  (None, 16, 16, 512)  0           ['conv3_block1_add[0][0]']       
                                                                                                  
 conv3_block2_1_conv (Conv2D)   (None, 16, 16, 128)  65664       ['conv3_block1_out[0][0]']       
                                                                                                  
 conv3_block2_1_bn (BatchNormal  (None, 16, 16, 128)  512        ['conv3_block2_1_conv[0][0]']    
 ization)                                                                                         
                                                                                                  
 conv3_block2_1_relu (Activatio  (None, 16, 16, 128)  0          ['conv3_block2_1_bn[0][0]']      
 n)                                                                                               
                                                                                                  
 conv3_block2_2_conv (Conv2D)   (None, 16, 16, 128)  147584      ['conv3_block2_1_relu[0][0]']    
                                                                                                  
 conv3_block2_2_bn (BatchNormal  (None, 16, 16, 128)  512        ['conv3_block2_2_conv[0][0]']    
 ization)                                                                                         
                                                                                                  
 conv3_block2_2_relu (Activatio  (None, 16, 16, 128)  0          ['conv3_block2_2_bn[0][0]']      
 n)                                                                                               
                                                                                                  
 conv3_block2_3_conv (Conv2D)   (None, 16, 16, 512)  66048       ['conv3_block2_2_relu[0][0]']    
                                                                                                  
 conv3_block2_3_bn (BatchNormal  (None, 16, 16, 512)  2048       ['conv3_block2_3_conv[0][0]']    
 ization)                                                                                         
                                                                                                  
 conv3_block2_add (Add)         (None, 16, 16, 512)  0           ['conv3_block1_out[0][0]',       
                                                                  'conv3_block2_3_bn[0][0]']      
                                                                                                  
 conv3_block2_out (Activation)  (None, 16, 16, 512)  0           ['conv3_block2_add[0][0]']       
                                                                                                  
 conv3_block3_1_conv (Conv2D)   (None, 16, 16, 128)  65664       ['conv3_block2_out[0][0]']       
                                                                                                  
 conv3_block3_1_bn (BatchNormal  (None, 16, 16, 128)  512        ['conv3_block3_1_conv[0][0]']    
 ization)                                                                                         
                                                                                                  
 conv3_block3_1_relu (Activatio  (None, 16, 16, 128)  0          ['conv3_block3_1_bn[0][0]']      
 n)                                                                                               
                                                                                                  
 conv3_block3_2_conv (Conv2D)   (None, 16, 16, 128)  147584      ['conv3_block3_1_relu[0][0]']    
                                                                                                  
 conv3_block3_2_bn (BatchNormal  (None, 16, 16, 128)  512        ['conv3_block3_2_conv[0][0]']    
 ization)                                                                                         
                                                                                                  
 conv3_block3_2_relu (Activatio  (None, 16, 16, 128)  0          ['conv3_block3_2_bn[0][0]']      
 n)                                                                                               
                                                                                                  
 conv3_block3_3_conv (Conv2D)   (None, 16, 16, 512)  66048       ['conv3_block3_2_relu[0][0]']    
                                                                                                  
 conv3_block3_3_bn (BatchNormal  (None, 16, 16, 512)  2048       ['conv3_block3_3_conv[0][0]']    
 ization)                                                                                         
                                                                                                  
 conv3_block3_add (Add)         (None, 16, 16, 512)  0           ['conv3_block2_out[0][0]',       
                                                                  'conv3_block3_3_bn[0][0]']      
                                                                                                  
 conv3_block3_out (Activation)  (None, 16, 16, 512)  0           ['conv3_block3_add[0][0]']       
                                                                                                  
 conv3_block4_1_conv (Conv2D)   (None, 16, 16, 128)  65664       ['conv3_block3_out[0][0]']       
                                                                                                  
 conv3_block4_1_bn (BatchNormal  (None, 16, 16, 128)  512        ['conv3_block4_1_conv[0][0]']    
 ization)                                                                                         
                                                                                                  
 conv3_block4_1_relu (Activatio  (None, 16, 16, 128)  0          ['conv3_block4_1_bn[0][0]']      
 n)                                                                                               
                                                                                                  
 conv3_block4_2_conv (Conv2D)   (None, 16, 16, 128)  147584      ['conv3_block4_1_relu[0][0]']    
                                                                                                  
 conv3_block4_2_bn (BatchNormal  (None, 16, 16, 128)  512        ['conv3_block4_2_conv[0][0]']    
 ization)                                                                                         
                                                                                                  
 conv3_block4_2_relu (Activatio  (None, 16, 16, 128)  0          ['conv3_block4_2_bn[0][0]']      
 n)                                                                                               
                                                                                                  
 conv3_block4_3_conv (Conv2D)   (None, 16, 16, 512)  66048       ['conv3_block4_2_relu[0][0]']    
                                                                                                  
 conv3_block4_3_bn (BatchNormal  (None, 16, 16, 512)  2048       ['conv3_block4_3_conv[0][0]']    
 ization)                                                                                         
                                                                                                  
 conv3_block4_add (Add)         (None, 16, 16, 512)  0           ['conv3_block3_out[0][0]',       
                                                                  'conv3_block4_3_bn[0][0]']      
                                                                                                  
 conv3_block4_out (Activation)  (None, 16, 16, 512)  0           ['conv3_block4_add[0][0]']       
                                                                                                  
 conv4_block1_1_conv (Conv2D)   (None, 8, 8, 256)    131328      ['conv3_block4_out[0][0]']       
                                                                                                  
 conv4_block1_1_bn (BatchNormal  (None, 8, 8, 256)   1024        ['conv4_block1_1_conv[0][0]']    
 ization)                                                                                         
                                                                                                  
 conv4_block1_1_relu (Activatio  (None, 8, 8, 256)   0           ['conv4_block1_1_bn[0][0]']      
 n)                                                                                               
                                                                                                  
 conv4_block1_2_conv (Conv2D)   (None, 8, 8, 256)    590080      ['conv4_block1_1_relu[0][0]']    
                                                                                                  
 conv4_block1_2_bn (BatchNormal  (None, 8, 8, 256)   1024        ['conv4_block1_2_conv[0][0]']    
 ization)                                                                                         
                                                                                                  
 conv4_block1_2_relu (Activatio  (None, 8, 8, 256)   0           ['conv4_block1_2_bn[0][0]']      
 n)                                                                                               
                                                                                                  
 conv4_block1_0_conv (Conv2D)   (None, 8, 8, 1024)   525312      ['conv3_block4_out[0][0]']       
                                                                                                  
 conv4_block1_3_conv (Conv2D)   (None, 8, 8, 1024)   263168      ['conv4_block1_2_relu[0][0]']    
                                                                                                  
 conv4_block1_0_bn (BatchNormal  (None, 8, 8, 1024)  4096        ['conv4_block1_0_conv[0][0]']    
 ization)                                                                                         
                                                                                                  
 conv4_block1_3_bn (BatchNormal  (None, 8, 8, 1024)  4096        ['conv4_block1_3_conv[0][0]']    
 ization)                                                                                         
                                                                                                  
 conv4_block1_add (Add)         (None, 8, 8, 1024)   0           ['conv4_block1_0_bn[0][0]',      
                                                                  'conv4_block1_3_bn[0][0]']      
                                                                                                  
 conv4_block1_out (Activation)  (None, 8, 8, 1024)   0           ['conv4_block1_add[0][0]']       
                                                                                                  
 conv4_block2_1_conv (Conv2D)   (None, 8, 8, 256)    262400      ['conv4_block1_out[0][0]']       
                                                                                                  
 conv4_block2_1_bn (BatchNormal  (None, 8, 8, 256)   1024        ['conv4_block2_1_conv[0][0]']    
 ization)                                                                                         
                                                                                                  
 conv4_block2_1_relu (Activatio  (None, 8, 8, 256)   0           ['conv4_block2_1_bn[0][0]']      
 n)                                                                                               
                                                                                                  
 conv4_block2_2_conv (Conv2D)   (None, 8, 8, 256)    590080      ['conv4_block2_1_relu[0][0]']    
                                                                                                  
 conv4_block2_2_bn (BatchNormal  (None, 8, 8, 256)   1024        ['conv4_block2_2_conv[0][0]']    
 ization)                                                                                         
                                                                                                  
 conv4_block2_2_relu (Activatio  (None, 8, 8, 256)   0           ['conv4_block2_2_bn[0][0]']      
 n)                                                                                               
                                                                                                  
 conv4_block2_3_conv (Conv2D)   (None, 8, 8, 1024)   263168      ['conv4_block2_2_relu[0][0]']    
                                                                                                  
 conv4_block2_3_bn (BatchNormal  (None, 8, 8, 1024)  4096        ['conv4_block2_3_conv[0][0]']    
 ization)                                                                                         
                                                                                                  
 conv4_block2_add (Add)         (None, 8, 8, 1024)   0           ['conv4_block1_out[0][0]',       
                                                                  'conv4_block2_3_bn[0][0]']      
                                                                                                  
 conv4_block2_out (Activation)  (None, 8, 8, 1024)   0           ['conv4_block2_add[0][0]']       
                                                                                                  
 conv4_block3_1_conv (Conv2D)   (None, 8, 8, 256)    262400      ['conv4_block2_out[0][0]']       
                                                                                                  
 conv4_block3_1_bn (BatchNormal  (None, 8, 8, 256)   1024        ['conv4_block3_1_conv[0][0]']    
 ization)                                                                                         
                                                                                                  
 conv4_block3_1_relu (Activatio  (None, 8, 8, 256)   0           ['conv4_block3_1_bn[0][0]']      
 n)                                                                                               
                                                                                                  
 conv4_block3_2_conv (Conv2D)   (None, 8, 8, 256)    590080      ['conv4_block3_1_relu[0][0]']    
                                                                                                  
 conv4_block3_2_bn (BatchNormal  (None, 8, 8, 256)   1024        ['conv4_block3_2_conv[0][0]']    
 ization)                                                                                         
                                                                                                  
 conv4_block3_2_relu (Activatio  (None, 8, 8, 256)   0           ['conv4_block3_2_bn[0][0]']      
 n)                                                                                               
                                                                                                  
 conv4_block3_3_conv (Conv2D)   (None, 8, 8, 1024)   263168      ['conv4_block3_2_relu[0][0]']    
                                                                                                  
 conv4_block3_3_bn (BatchNormal  (None, 8, 8, 1024)  4096        ['conv4_block3_3_conv[0][0]']    
 ization)                                                                                         
                                                                                                  
 conv4_block3_add (Add)         (None, 8, 8, 1024)   0           ['conv4_block2_out[0][0]',       
                                                                  'conv4_block3_3_bn[0][0]']      
                                                                                                  
 conv4_block3_out (Activation)  (None, 8, 8, 1024)   0           ['conv4_block3_add[0][0]']       
                                                                                                  
 conv4_block4_1_conv (Conv2D)   (None, 8, 8, 256)    262400      ['conv4_block3_out[0][0]']       
                                                                                                  
 conv4_block4_1_bn (BatchNormal  (None, 8, 8, 256)   1024        ['conv4_block4_1_conv[0][0]']    
 ization)                                                                                         
                                                                                                  
 conv4_block4_1_relu (Activatio  (None, 8, 8, 256)   0           ['conv4_block4_1_bn[0][0]']      
 n)                                                                                               
                                                                                                  
 conv4_block4_2_conv (Conv2D)   (None, 8, 8, 256)    590080      ['conv4_block4_1_relu[0][0]']    
                                                                                                  
 conv4_block4_2_bn (BatchNormal  (None, 8, 8, 256)   1024        ['conv4_block4_2_conv[0][0]']    
 ization)                                                                                         
                                                                                                  
 conv4_block4_2_relu (Activatio  (None, 8, 8, 256)   0           ['conv4_block4_2_bn[0][0]']      
 n)                                                                                               
                                                                                                  
 conv4_block4_3_conv (Conv2D)   (None, 8, 8, 1024)   263168      ['conv4_block4_2_relu[0][0]']    
                                                                                                  
 conv4_block4_3_bn (BatchNormal  (None, 8, 8, 1024)  4096        ['conv4_block4_3_conv[0][0]']    
 ization)                                                                                         
                                                                                                  
 conv4_block4_add (Add)         (None, 8, 8, 1024)   0           ['conv4_block3_out[0][0]',       
                                                                  'conv4_block4_3_bn[0][0]']      
                                                                                                  
 conv4_block4_out (Activation)  (None, 8, 8, 1024)   0           ['conv4_block4_add[0][0]']       
                                                                                                  
 conv4_block5_1_conv (Conv2D)   (None, 8, 8, 256)    262400      ['conv4_block4_out[0][0]']       
                                                                                                  
 conv4_block5_1_bn (BatchNormal  (None, 8, 8, 256)   1024        ['conv4_block5_1_conv[0][0]']    
 ization)                                                                                         
                                                                                                  
 conv4_block5_1_relu (Activatio  (None, 8, 8, 256)   0           ['conv4_block5_1_bn[0][0]']      
 n)                                                                                               
                                                                                                  
 conv4_block5_2_conv (Conv2D)   (None, 8, 8, 256)    590080      ['conv4_block5_1_relu[0][0]']    
                                                                                                  
 conv4_block5_2_bn (BatchNormal  (None, 8, 8, 256)   1024        ['conv4_block5_2_conv[0][0]']    
 ization)                                                                                         
                                                                                                  
 conv4_block5_2_relu (Activatio  (None, 8, 8, 256)   0           ['conv4_block5_2_bn[0][0]']      
 n)                                                                                               
                                                                                                  
 conv4_block5_3_conv (Conv2D)   (None, 8, 8, 1024)   263168      ['conv4_block5_2_relu[0][0]']    
                                                                                                  
 conv4_block5_3_bn (BatchNormal  (None, 8, 8, 1024)  4096        ['conv4_block5_3_conv[0][0]']    
 ization)                                                                                         
                                                                                                  
 conv4_block5_add (Add)         (None, 8, 8, 1024)   0           ['conv4_block4_out[0][0]',       
                                                                  'conv4_block5_3_bn[0][0]']      
                                                                                                  
 conv4_block5_out (Activation)  (None, 8, 8, 1024)   0           ['conv4_block5_add[0][0]']       
                                                                                                  
 conv4_block6_1_conv (Conv2D)   (None, 8, 8, 256)    262400      ['conv4_block5_out[0][0]']       
                                                                                                  
 conv4_block6_1_bn (BatchNormal  (None, 8, 8, 256)   1024        ['conv4_block6_1_conv[0][0]']    
 ization)                                                                                         
                                                                                                  
 conv4_block6_1_relu (Activatio  (None, 8, 8, 256)   0           ['conv4_block6_1_bn[0][0]']      
 n)                                                                                               
                                                                                                  
 conv4_block6_2_conv (Conv2D)   (None, 8, 8, 256)    590080      ['conv4_block6_1_relu[0][0]']    
                                                                                                  
 conv4_block6_2_bn (BatchNormal  (None, 8, 8, 256)   1024        ['conv4_block6_2_conv[0][0]']    
 ization)                                                                                         
                                                                                                  
 conv4_block6_2_relu (Activatio  (None, 8, 8, 256)   0           ['conv4_block6_2_bn[0][0]']      
 n)                                                                                               
                                                                                                  
 conv4_block6_3_conv (Conv2D)   (None, 8, 8, 1024)   263168      ['conv4_block6_2_relu[0][0]']    
                                                                                                  
 conv4_block6_3_bn (BatchNormal  (None, 8, 8, 1024)  4096        ['conv4_block6_3_conv[0][0]']    
 ization)                                                                                         
                                                                                                  
 conv4_block6_add (Add)         (None, 8, 8, 1024)   0           ['conv4_block5_out[0][0]',       
                                                                  'conv4_block6_3_bn[0][0]']      
                                                                                                  
 conv4_block6_out (Activation)  (None, 8, 8, 1024)   0           ['conv4_block6_add[0][0]']       
                                                                                                  
 conv5_block1_1_conv (Conv2D)   (None, 4, 4, 512)    524800      ['conv4_block6_out[0][0]']       
                                                                                                  
 conv5_block1_1_bn (BatchNormal  (None, 4, 4, 512)   2048        ['conv5_block1_1_conv[0][0]']    
 ization)                                                                                         
                                                                                                  
 conv5_block1_1_relu (Activatio  (None, 4, 4, 512)   0           ['conv5_block1_1_bn[0][0]']      
 n)                                                                                               
                                                                                                  
 conv5_block1_2_conv (Conv2D)   (None, 4, 4, 512)    2359808     ['conv5_block1_1_relu[0][0]']    
                                                                                                  
 conv5_block1_2_bn (BatchNormal  (None, 4, 4, 512)   2048        ['conv5_block1_2_conv[0][0]']    
 ization)                                                                                         
                                                                                                  
 conv5_block1_2_relu (Activatio  (None, 4, 4, 512)   0           ['conv5_block1_2_bn[0][0]']      
 n)                                                                                               
                                                                                                  
 conv5_block1_0_conv (Conv2D)   (None, 4, 4, 2048)   2099200     ['conv4_block6_out[0][0]']       
                                                                                                  
 conv5_block1_3_conv (Conv2D)   (None, 4, 4, 2048)   1050624     ['conv5_block1_2_relu[0][0]']    
                                                                                                  
 conv5_block1_0_bn (BatchNormal  (None, 4, 4, 2048)  8192        ['conv5_block1_0_conv[0][0]']    
 ization)                                                                                         
                                                                                                  
 conv5_block1_3_bn (BatchNormal  (None, 4, 4, 2048)  8192        ['conv5_block1_3_conv[0][0]']    
 ization)                                                                                         
                                                                                                  
 conv5_block1_add (Add)         (None, 4, 4, 2048)   0           ['conv5_block1_0_bn[0][0]',      
                                                                  'conv5_block1_3_bn[0][0]']      
                                                                                                  
 conv5_block1_out (Activation)  (None, 4, 4, 2048)   0           ['conv5_block1_add[0][0]']       
                                                                                                  
 conv5_block2_1_conv (Conv2D)   (None, 4, 4, 512)    1049088     ['conv5_block1_out[0][0]']       
                                                                                                  
 conv5_block2_1_bn (BatchNormal  (None, 4, 4, 512)   2048        ['conv5_block2_1_conv[0][0]']    
 ization)                                                                                         
                                                                                                  
 conv5_block2_1_relu (Activatio  (None, 4, 4, 512)   0           ['conv5_block2_1_bn[0][0]']      
 n)                                                                                               
                                                                                                  
 conv5_block2_2_conv (Conv2D)   (None, 4, 4, 512)    2359808     ['conv5_block2_1_relu[0][0]']    
                                                                                                  
 conv5_block2_2_bn (BatchNormal  (None, 4, 4, 512)   2048        ['conv5_block2_2_conv[0][0]']    
 ization)                                                                                         
                                                                                                  
 conv5_block2_2_relu (Activatio  (None, 4, 4, 512)   0           ['conv5_block2_2_bn[0][0]']      
 n)                                                                                               
                                                                                                  
 conv5_block2_3_conv (Conv2D)   (None, 4, 4, 2048)   1050624     ['conv5_block2_2_relu[0][0]']    
                                                                                                  
 conv5_block2_3_bn (BatchNormal  (None, 4, 4, 2048)  8192        ['conv5_block2_3_conv[0][0]']    
 ization)                                                                                         
                                                                                                  
 conv5_block2_add (Add)         (None, 4, 4, 2048)   0           ['conv5_block1_out[0][0]',       
                                                                  'conv5_block2_3_bn[0][0]']      
                                                                                                  
 conv5_block2_out (Activation)  (None, 4, 4, 2048)   0           ['conv5_block2_add[0][0]']       
                                                                                                  
 conv5_block3_1_conv (Conv2D)   (None, 4, 4, 512)    1049088     ['conv5_block2_out[0][0]']       
                                                                                                  
 conv5_block3_1_bn (BatchNormal  (None, 4, 4, 512)   2048        ['conv5_block3_1_conv[0][0]']    
 ization)                                                                                         
                                                                                                  
 conv5_block3_1_relu (Activatio  (None, 4, 4, 512)   0           ['conv5_block3_1_bn[0][0]']      
 n)                                                                                               
                                                                                                  
 conv5_block3_2_conv (Conv2D)   (None, 4, 4, 512)    2359808     ['conv5_block3_1_relu[0][0]']    
                                                                                                  
 conv5_block3_2_bn (BatchNormal  (None, 4, 4, 512)   2048        ['conv5_block3_2_conv[0][0]']    
 ization)                                                                                         
                                                                                                  
 conv5_block3_2_relu (Activatio  (None, 4, 4, 512)   0           ['conv5_block3_2_bn[0][0]']      
 n)                                                                                               
                                                                                                  
 conv5_block3_3_conv (Conv2D)   (None, 4, 4, 2048)   1050624     ['conv5_block3_2_relu[0][0]']    
                                                                                                  
 conv5_block3_3_bn (BatchNormal  (None, 4, 4, 2048)  8192        ['conv5_block3_3_conv[0][0]']    
 ization)                                                                                         
                                                                                                  
 conv5_block3_add (Add)         (None, 4, 4, 2048)   0           ['conv5_block2_out[0][0]',       
                                                                  'conv5_block3_3_bn[0][0]']      
                                                                                                  
 conv5_block3_out (Activation)  (None, 4, 4, 2048)   0           ['conv5_block3_add[0][0]']       
                                                                                                  
 avg_pool (GlobalAveragePooling  (None, 2048)        0           ['conv5_block3_out[0][0]']       
 2D)                                                                                              
                                                                                                  
==================================================================================================
Total params: 23,581,440
Trainable params: 23,528,320
Non-trainable params: 53,120
__________________________________________________________________________________________________

How are we gonna to modify it to suit our project??

• Reducing the amount of filters :
  • A regular Resnet50 has around 23 million trainable parameters, by reducing the filters, we can have a trainable model that does not take up too much time
• Normalizing our inputs :
  • This helps our model converge faster and help it to generalise better
• Network Depth :
  • For the 31x31 input size, a 12-layered ResNet will be implemented, and for the 128x128 input size, a 16-layered ResNet will be used. As we require deeper models for capture more complex features in 128x128
• Including dropout layer:
  • To further combat overfitting, a dropout layer will be incorporated.
• L2 regularization:
  • L2 regularization will be applied to the network's weights. This regularization technique adds a penalty term to the loss function, discouraging large weight values.

def identity_block(X, f, filters):
    F1, F2, F3 = filters

    X_shortcut = X

    X = Conv2D(filters=F1, kernel_size=(1, 1), strides=(1, 1), padding='valid')(X)
    X = BatchNormalization(axis=3)(X)
    X = Activation('relu')(X)

    X = Conv2D(filters=F2, kernel_size=(f, f), strides=(1, 1), padding='same')(X)
    X = BatchNormalization(axis=3)(X)
    X = Activation('relu')(X)

    X = Conv2D(filters=F3, kernel_size=(1, 1), strides=(1, 1), padding='valid')(X)
    X = BatchNormalization(axis=3)(X)

    X = Add()([X, X_shortcut])
    X = Activation('relu')(X)

    return X

def projection_block(X, f, filters, s=2):
    F1, F2, F3 = filters

    X_shortcut = X

    X = Conv2D(F1, (1, 1), strides=(s, s))(X)
    X = BatchNormalization(axis=3)(X)
    X = Activation('relu')(X)

    X = Conv2D(F2, (f, f), strides=(1, 1), padding='same')(X)
    X = BatchNormalization(axis=3)(X)
    X = Activation('relu')(X)

    X = Conv2D(F3, (1, 1), strides=(1, 1), padding='valid')(X)
    X = BatchNormalization(axis=3)(X)

    X_shortcut = Conv2D(F3, (1, 1), strides=(s, s))(X_shortcut)
    X_shortcut = BatchNormalization(axis=3)(X_shortcut)

    X = Add()([X, X_shortcut])
    X = Activation('relu')(X)
    return X

def CustomResnet(input_shape, classes=15):
    tf.keras.backend.clear_session()
    X_input = Input(input_shape)

    # Stage 1
    X = Conv2D(64, (7, 7), strides=(2, 2), padding='same')(X_input)
    X = BatchNormalization(axis=3)(X)
    X = Activation('relu')(X)
    X = MaxPooling2D((3, 3), strides=(2, 2))(X)

    if input_shape == (31, 31, 1):
        stages = [
            {'blocks': 3, 'filters': [8, 8, 32]},
            {'blocks': 3, 'filters': [16, 16, 64]},
            {'blocks': 3, 'filters': [32, 32, 128]},
            {'blocks': 3, 'filters': [64, 64, 256]}
        ]
    elif input_shape == (128, 128, 1):
        stages = [
            {'blocks': 4, 'filters': [8, 8, 32]},
            {'blocks': 4, 'filters': [16, 16, 64]},
            {'blocks': 4, 'filters': [32, 32, 128]},
            {'blocks': 4, 'filters': [64, 64, 256]}
        ]
    for i, stage in enumerate(stages):
        X = projection_block(X, f=3, filters=stage['filters'], s=1 if i == 0 else 2)
        for _ in range(stage['blocks']):
            X = identity_block(X, 3, stage['filters'])

    X = AveragePooling2D(pool_size=(2, 2), padding='same')(X)
    X = Flatten()(X)
    X = Dense(256, activation='relu',kernel_regularizer=l2(weight_decay))(X)
    X = Dense(256, activation='relu',kernel_regularizer=l2(weight_decay))(X)
    X = Dropout(0.2)(X)
    X = Dense(classes, activation='softmax')(X)

    model = Model(inputs=X_input, outputs=X, name='ResNet50')

    return model


CustomResnet_128 = CustomResnet(input_shape=(128, 128, 1), classes=15)
CustomResnet_31 = CustomResnet(input_shape=(31, 31, 1), classes=15)

CustomResnet_128.compile(optimizer=SGD(learning_rate=learning_rate, momentum=momentum), loss='categorical_crossentropy', metrics=['accuracy',f1_m])
CustomResnet_31.compile(optimizer=SGD(learning_rate=learning_rate, momentum=momentum), loss='categorical_crossentropy', metrics=['accuracy',f1_m])

CustomResnet_128.summary()

Model: "ResNet50"
__________________________________________________________________________________________________
 Layer (type)                   Output Shape         Param #     Connected to                     
==================================================================================================
 input_1 (InputLayer)           [(None, 128, 128, 1  0           []                               
                                )]                                                                
                                                                                                  
 conv2d (Conv2D)                (None, 64, 64, 64)   3200        ['input_1[0][0]']                
                                                                                                  
 batch_normalization (BatchNorm  (None, 64, 64, 64)  256         ['conv2d[0][0]']                 
 alization)                                                                                       
                                                                                                  
 activation (Activation)        (None, 64, 64, 64)   0           ['batch_normalization[0][0]']    
                                                                                                  
 max_pooling2d (MaxPooling2D)   (None, 31, 31, 64)   0           ['activation[0][0]']             
                                                                                                  
 conv2d_1 (Conv2D)              (None, 31, 31, 8)    520         ['max_pooling2d[0][0]']          
                                                                                                  
 batch_normalization_1 (BatchNo  (None, 31, 31, 8)   32          ['conv2d_1[0][0]']               
 rmalization)                                                                                     
                                                                                                  
 activation_1 (Activation)      (None, 31, 31, 8)    0           ['batch_normalization_1[0][0]']  
                                                                                                  
 conv2d_2 (Conv2D)              (None, 31, 31, 8)    584         ['activation_1[0][0]']           
                                                                                                  
 batch_normalization_2 (BatchNo  (None, 31, 31, 8)   32          ['conv2d_2[0][0]']               
 rmalization)                                                                                     
                                                                                                  
 activation_2 (Activation)      (None, 31, 31, 8)    0           ['batch_normalization_2[0][0]']  
                                                                                                  
 conv2d_3 (Conv2D)              (None, 31, 31, 32)   288         ['activation_2[0][0]']           
                                                                                                  
 conv2d_4 (Conv2D)              (None, 31, 31, 32)   2080        ['max_pooling2d[0][0]']          
                                                                                                  
 batch_normalization_3 (BatchNo  (None, 31, 31, 32)  128         ['conv2d_3[0][0]']               
 rmalization)                                                                                     
                                                                                                  
 batch_normalization_4 (BatchNo  (None, 31, 31, 32)  128         ['conv2d_4[0][0]']               
 rmalization)                                                                                     
                                                                                                  
 add (Add)                      (None, 31, 31, 32)   0           ['batch_normalization_3[0][0]',  
                                                                  'batch_normalization_4[0][0]']  
                                                                                                  
 activation_3 (Activation)      (None, 31, 31, 32)   0           ['add[0][0]']                    
                                                                                                  
 conv2d_5 (Conv2D)              (None, 31, 31, 8)    264         ['activation_3[0][0]']           
                                                                                                  
 batch_normalization_5 (BatchNo  (None, 31, 31, 8)   32          ['conv2d_5[0][0]']               
 rmalization)                                                                                     
                                                                                                  
 activation_4 (Activation)      (None, 31, 31, 8)    0           ['batch_normalization_5[0][0]']  
                                                                                                  
 conv2d_6 (Conv2D)              (None, 31, 31, 8)    584         ['activation_4[0][0]']           
                                                                                                  
 batch_normalization_6 (BatchNo  (None, 31, 31, 8)   32          ['conv2d_6[0][0]']               
 rmalization)                                                                                     
                                                                                                  
 activation_5 (Activation)      (None, 31, 31, 8)    0           ['batch_normalization_6[0][0]']  
                                                                                                  
 conv2d_7 (Conv2D)              (None, 31, 31, 32)   288         ['activation_5[0][0]']           
                                                                                                  
 batch_normalization_7 (BatchNo  (None, 31, 31, 32)  128         ['conv2d_7[0][0]']               
 rmalization)                                                                                     
                                                                                                  
 add_1 (Add)                    (None, 31, 31, 32)   0           ['batch_normalization_7[0][0]',  
                                                                  'activation_3[0][0]']           
                                                                                                  
 activation_6 (Activation)      (None, 31, 31, 32)   0           ['add_1[0][0]']                  
                                                                                                  
 conv2d_8 (Conv2D)              (None, 31, 31, 8)    264         ['activation_6[0][0]']           
                                                                                                  
 batch_normalization_8 (BatchNo  (None, 31, 31, 8)   32          ['conv2d_8[0][0]']               
 rmalization)                                                                                     
                                                                                                  
 activation_7 (Activation)      (None, 31, 31, 8)    0           ['batch_normalization_8[0][0]']  
                                                                                                  
 conv2d_9 (Conv2D)              (None, 31, 31, 8)    584         ['activation_7[0][0]']           
                                                                                                  
 batch_normalization_9 (BatchNo  (None, 31, 31, 8)   32          ['conv2d_9[0][0]']               
 rmalization)                                                                                     
                                                                                                  
 activation_8 (Activation)      (None, 31, 31, 8)    0           ['batch_normalization_9[0][0]']  
                                                                                                  
 conv2d_10 (Conv2D)             (None, 31, 31, 32)   288         ['activation_8[0][0]']           
                                                                                                  
 batch_normalization_10 (BatchN  (None, 31, 31, 32)  128         ['conv2d_10[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 add_2 (Add)                    (None, 31, 31, 32)   0           ['batch_normalization_10[0][0]', 
                                                                  'activation_6[0][0]']           
                                                                                                  
 activation_9 (Activation)      (None, 31, 31, 32)   0           ['add_2[0][0]']                  
                                                                                                  
 conv2d_11 (Conv2D)             (None, 31, 31, 8)    264         ['activation_9[0][0]']           
                                                                                                  
 batch_normalization_11 (BatchN  (None, 31, 31, 8)   32          ['conv2d_11[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_10 (Activation)     (None, 31, 31, 8)    0           ['batch_normalization_11[0][0]'] 
                                                                                                  
 conv2d_12 (Conv2D)             (None, 31, 31, 8)    584         ['activation_10[0][0]']          
                                                                                                  
 batch_normalization_12 (BatchN  (None, 31, 31, 8)   32          ['conv2d_12[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_11 (Activation)     (None, 31, 31, 8)    0           ['batch_normalization_12[0][0]'] 
                                                                                                  
 conv2d_13 (Conv2D)             (None, 31, 31, 32)   288         ['activation_11[0][0]']          
                                                                                                  
 batch_normalization_13 (BatchN  (None, 31, 31, 32)  128         ['conv2d_13[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 add_3 (Add)                    (None, 31, 31, 32)   0           ['batch_normalization_13[0][0]', 
                                                                  'activation_9[0][0]']           
                                                                                                  
 activation_12 (Activation)     (None, 31, 31, 32)   0           ['add_3[0][0]']                  
                                                                                                  
 conv2d_14 (Conv2D)             (None, 31, 31, 8)    264         ['activation_12[0][0]']          
                                                                                                  
 batch_normalization_14 (BatchN  (None, 31, 31, 8)   32          ['conv2d_14[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_13 (Activation)     (None, 31, 31, 8)    0           ['batch_normalization_14[0][0]'] 
                                                                                                  
 conv2d_15 (Conv2D)             (None, 31, 31, 8)    584         ['activation_13[0][0]']          
                                                                                                  
 batch_normalization_15 (BatchN  (None, 31, 31, 8)   32          ['conv2d_15[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_14 (Activation)     (None, 31, 31, 8)    0           ['batch_normalization_15[0][0]'] 
                                                                                                  
 conv2d_16 (Conv2D)             (None, 31, 31, 32)   288         ['activation_14[0][0]']          
                                                                                                  
 batch_normalization_16 (BatchN  (None, 31, 31, 32)  128         ['conv2d_16[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 add_4 (Add)                    (None, 31, 31, 32)   0           ['batch_normalization_16[0][0]', 
                                                                  'activation_12[0][0]']          
                                                                                                  
 activation_15 (Activation)     (None, 31, 31, 32)   0           ['add_4[0][0]']                  
                                                                                                  
 conv2d_17 (Conv2D)             (None, 16, 16, 16)   528         ['activation_15[0][0]']          
                                                                                                  
 batch_normalization_17 (BatchN  (None, 16, 16, 16)  64          ['conv2d_17[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_16 (Activation)     (None, 16, 16, 16)   0           ['batch_normalization_17[0][0]'] 
                                                                                                  
 conv2d_18 (Conv2D)             (None, 16, 16, 16)   2320        ['activation_16[0][0]']          
                                                                                                  
 batch_normalization_18 (BatchN  (None, 16, 16, 16)  64          ['conv2d_18[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_17 (Activation)     (None, 16, 16, 16)   0           ['batch_normalization_18[0][0]'] 
                                                                                                  
 conv2d_19 (Conv2D)             (None, 16, 16, 64)   1088        ['activation_17[0][0]']          
                                                                                                  
 conv2d_20 (Conv2D)             (None, 16, 16, 64)   2112        ['activation_15[0][0]']          
                                                                                                  
 batch_normalization_19 (BatchN  (None, 16, 16, 64)  256         ['conv2d_19[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 batch_normalization_20 (BatchN  (None, 16, 16, 64)  256         ['conv2d_20[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 add_5 (Add)                    (None, 16, 16, 64)   0           ['batch_normalization_19[0][0]', 
                                                                  'batch_normalization_20[0][0]'] 
                                                                                                  
 activation_18 (Activation)     (None, 16, 16, 64)   0           ['add_5[0][0]']                  
                                                                                                  
 conv2d_21 (Conv2D)             (None, 16, 16, 16)   1040        ['activation_18[0][0]']          
                                                                                                  
 batch_normalization_21 (BatchN  (None, 16, 16, 16)  64          ['conv2d_21[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_19 (Activation)     (None, 16, 16, 16)   0           ['batch_normalization_21[0][0]'] 
                                                                                                  
 conv2d_22 (Conv2D)             (None, 16, 16, 16)   2320        ['activation_19[0][0]']          
                                                                                                  
 batch_normalization_22 (BatchN  (None, 16, 16, 16)  64          ['conv2d_22[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_20 (Activation)     (None, 16, 16, 16)   0           ['batch_normalization_22[0][0]'] 
                                                                                                  
 conv2d_23 (Conv2D)             (None, 16, 16, 64)   1088        ['activation_20[0][0]']          
                                                                                                  
 batch_normalization_23 (BatchN  (None, 16, 16, 64)  256         ['conv2d_23[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 add_6 (Add)                    (None, 16, 16, 64)   0           ['batch_normalization_23[0][0]', 
                                                                  'activation_18[0][0]']          
                                                                                                  
 activation_21 (Activation)     (None, 16, 16, 64)   0           ['add_6[0][0]']                  
                                                                                                  
 conv2d_24 (Conv2D)             (None, 16, 16, 16)   1040        ['activation_21[0][0]']          
                                                                                                  
 batch_normalization_24 (BatchN  (None, 16, 16, 16)  64          ['conv2d_24[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_22 (Activation)     (None, 16, 16, 16)   0           ['batch_normalization_24[0][0]'] 
                                                                                                  
 conv2d_25 (Conv2D)             (None, 16, 16, 16)   2320        ['activation_22[0][0]']          
                                                                                                  
 batch_normalization_25 (BatchN  (None, 16, 16, 16)  64          ['conv2d_25[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_23 (Activation)     (None, 16, 16, 16)   0           ['batch_normalization_25[0][0]'] 
                                                                                                  
 conv2d_26 (Conv2D)             (None, 16, 16, 64)   1088        ['activation_23[0][0]']          
                                                                                                  
 batch_normalization_26 (BatchN  (None, 16, 16, 64)  256         ['conv2d_26[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 add_7 (Add)                    (None, 16, 16, 64)   0           ['batch_normalization_26[0][0]', 
                                                                  'activation_21[0][0]']          
                                                                                                  
 activation_24 (Activation)     (None, 16, 16, 64)   0           ['add_7[0][0]']                  
                                                                                                  
 conv2d_27 (Conv2D)             (None, 16, 16, 16)   1040        ['activation_24[0][0]']          
                                                                                                  
 batch_normalization_27 (BatchN  (None, 16, 16, 16)  64          ['conv2d_27[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_25 (Activation)     (None, 16, 16, 16)   0           ['batch_normalization_27[0][0]'] 
                                                                                                  
 conv2d_28 (Conv2D)             (None, 16, 16, 16)   2320        ['activation_25[0][0]']          
                                                                                                  
 batch_normalization_28 (BatchN  (None, 16, 16, 16)  64          ['conv2d_28[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_26 (Activation)     (None, 16, 16, 16)   0           ['batch_normalization_28[0][0]'] 
                                                                                                  
 conv2d_29 (Conv2D)             (None, 16, 16, 64)   1088        ['activation_26[0][0]']          
                                                                                                  
 batch_normalization_29 (BatchN  (None, 16, 16, 64)  256         ['conv2d_29[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 add_8 (Add)                    (None, 16, 16, 64)   0           ['batch_normalization_29[0][0]', 
                                                                  'activation_24[0][0]']          
                                                                                                  
 activation_27 (Activation)     (None, 16, 16, 64)   0           ['add_8[0][0]']                  
                                                                                                  
 conv2d_30 (Conv2D)             (None, 16, 16, 16)   1040        ['activation_27[0][0]']          
                                                                                                  
 batch_normalization_30 (BatchN  (None, 16, 16, 16)  64          ['conv2d_30[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_28 (Activation)     (None, 16, 16, 16)   0           ['batch_normalization_30[0][0]'] 
                                                                                                  
 conv2d_31 (Conv2D)             (None, 16, 16, 16)   2320        ['activation_28[0][0]']          
                                                                                                  
 batch_normalization_31 (BatchN  (None, 16, 16, 16)  64          ['conv2d_31[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_29 (Activation)     (None, 16, 16, 16)   0           ['batch_normalization_31[0][0]'] 
                                                                                                  
 conv2d_32 (Conv2D)             (None, 16, 16, 64)   1088        ['activation_29[0][0]']          
                                                                                                  
 batch_normalization_32 (BatchN  (None, 16, 16, 64)  256         ['conv2d_32[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 add_9 (Add)                    (None, 16, 16, 64)   0           ['batch_normalization_32[0][0]', 
                                                                  'activation_27[0][0]']          
                                                                                                  
 activation_30 (Activation)     (None, 16, 16, 64)   0           ['add_9[0][0]']                  
                                                                                                  
 conv2d_33 (Conv2D)             (None, 8, 8, 32)     2080        ['activation_30[0][0]']          
                                                                                                  
 batch_normalization_33 (BatchN  (None, 8, 8, 32)    128         ['conv2d_33[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_31 (Activation)     (None, 8, 8, 32)     0           ['batch_normalization_33[0][0]'] 
                                                                                                  
 conv2d_34 (Conv2D)             (None, 8, 8, 32)     9248        ['activation_31[0][0]']          
                                                                                                  
 batch_normalization_34 (BatchN  (None, 8, 8, 32)    128         ['conv2d_34[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_32 (Activation)     (None, 8, 8, 32)     0           ['batch_normalization_34[0][0]'] 
                                                                                                  
 conv2d_35 (Conv2D)             (None, 8, 8, 128)    4224        ['activation_32[0][0]']          
                                                                                                  
 conv2d_36 (Conv2D)             (None, 8, 8, 128)    8320        ['activation_30[0][0]']          
                                                                                                  
 batch_normalization_35 (BatchN  (None, 8, 8, 128)   512         ['conv2d_35[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 batch_normalization_36 (BatchN  (None, 8, 8, 128)   512         ['conv2d_36[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 add_10 (Add)                   (None, 8, 8, 128)    0           ['batch_normalization_35[0][0]', 
                                                                  'batch_normalization_36[0][0]'] 
                                                                                                  
 activation_33 (Activation)     (None, 8, 8, 128)    0           ['add_10[0][0]']                 
                                                                                                  
 conv2d_37 (Conv2D)             (None, 8, 8, 32)     4128        ['activation_33[0][0]']          
                                                                                                  
 batch_normalization_37 (BatchN  (None, 8, 8, 32)    128         ['conv2d_37[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_34 (Activation)     (None, 8, 8, 32)     0           ['batch_normalization_37[0][0]'] 
                                                                                                  
 conv2d_38 (Conv2D)             (None, 8, 8, 32)     9248        ['activation_34[0][0]']          
                                                                                                  
 batch_normalization_38 (BatchN  (None, 8, 8, 32)    128         ['conv2d_38[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_35 (Activation)     (None, 8, 8, 32)     0           ['batch_normalization_38[0][0]'] 
                                                                                                  
 conv2d_39 (Conv2D)             (None, 8, 8, 128)    4224        ['activation_35[0][0]']          
                                                                                                  
 batch_normalization_39 (BatchN  (None, 8, 8, 128)   512         ['conv2d_39[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 add_11 (Add)                   (None, 8, 8, 128)    0           ['batch_normalization_39[0][0]', 
                                                                  'activation_33[0][0]']          
                                                                                                  
 activation_36 (Activation)     (None, 8, 8, 128)    0           ['add_11[0][0]']                 
                                                                                                  
 conv2d_40 (Conv2D)             (None, 8, 8, 32)     4128        ['activation_36[0][0]']          
                                                                                                  
 batch_normalization_40 (BatchN  (None, 8, 8, 32)    128         ['conv2d_40[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_37 (Activation)     (None, 8, 8, 32)     0           ['batch_normalization_40[0][0]'] 
                                                                                                  
 conv2d_41 (Conv2D)             (None, 8, 8, 32)     9248        ['activation_37[0][0]']          
                                                                                                  
 batch_normalization_41 (BatchN  (None, 8, 8, 32)    128         ['conv2d_41[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_38 (Activation)     (None, 8, 8, 32)     0           ['batch_normalization_41[0][0]'] 
                                                                                                  
 conv2d_42 (Conv2D)             (None, 8, 8, 128)    4224        ['activation_38[0][0]']          
                                                                                                  
 batch_normalization_42 (BatchN  (None, 8, 8, 128)   512         ['conv2d_42[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 add_12 (Add)                   (None, 8, 8, 128)    0           ['batch_normalization_42[0][0]', 
                                                                  'activation_36[0][0]']          
                                                                                                  
 activation_39 (Activation)     (None, 8, 8, 128)    0           ['add_12[0][0]']                 
                                                                                                  
 conv2d_43 (Conv2D)             (None, 8, 8, 32)     4128        ['activation_39[0][0]']          
                                                                                                  
 batch_normalization_43 (BatchN  (None, 8, 8, 32)    128         ['conv2d_43[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_40 (Activation)     (None, 8, 8, 32)     0           ['batch_normalization_43[0][0]'] 
                                                                                                  
 conv2d_44 (Conv2D)             (None, 8, 8, 32)     9248        ['activation_40[0][0]']          
                                                                                                  
 batch_normalization_44 (BatchN  (None, 8, 8, 32)    128         ['conv2d_44[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_41 (Activation)     (None, 8, 8, 32)     0           ['batch_normalization_44[0][0]'] 
                                                                                                  
 conv2d_45 (Conv2D)             (None, 8, 8, 128)    4224        ['activation_41[0][0]']          
                                                                                                  
 batch_normalization_45 (BatchN  (None, 8, 8, 128)   512         ['conv2d_45[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 add_13 (Add)                   (None, 8, 8, 128)    0           ['batch_normalization_45[0][0]', 
                                                                  'activation_39[0][0]']          
                                                                                                  
 activation_42 (Activation)     (None, 8, 8, 128)    0           ['add_13[0][0]']                 
                                                                                                  
 conv2d_46 (Conv2D)             (None, 8, 8, 32)     4128        ['activation_42[0][0]']          
                                                                                                  
 batch_normalization_46 (BatchN  (None, 8, 8, 32)    128         ['conv2d_46[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_43 (Activation)     (None, 8, 8, 32)     0           ['batch_normalization_46[0][0]'] 
                                                                                                  
 conv2d_47 (Conv2D)             (None, 8, 8, 32)     9248        ['activation_43[0][0]']          
                                                                                                  
 batch_normalization_47 (BatchN  (None, 8, 8, 32)    128         ['conv2d_47[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_44 (Activation)     (None, 8, 8, 32)     0           ['batch_normalization_47[0][0]'] 
                                                                                                  
 conv2d_48 (Conv2D)             (None, 8, 8, 128)    4224        ['activation_44[0][0]']          
                                                                                                  
 batch_normalization_48 (BatchN  (None, 8, 8, 128)   512         ['conv2d_48[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 add_14 (Add)                   (None, 8, 8, 128)    0           ['batch_normalization_48[0][0]', 
                                                                  'activation_42[0][0]']          
                                                                                                  
 activation_45 (Activation)     (None, 8, 8, 128)    0           ['add_14[0][0]']                 
                                                                                                  
 conv2d_49 (Conv2D)             (None, 4, 4, 64)     8256        ['activation_45[0][0]']          
                                                                                                  
 batch_normalization_49 (BatchN  (None, 4, 4, 64)    256         ['conv2d_49[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_46 (Activation)     (None, 4, 4, 64)     0           ['batch_normalization_49[0][0]'] 
                                                                                                  
 conv2d_50 (Conv2D)             (None, 4, 4, 64)     36928       ['activation_46[0][0]']          
                                                                                                  
 batch_normalization_50 (BatchN  (None, 4, 4, 64)    256         ['conv2d_50[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_47 (Activation)     (None, 4, 4, 64)     0           ['batch_normalization_50[0][0]'] 
                                                                                                  
 conv2d_51 (Conv2D)             (None, 4, 4, 256)    16640       ['activation_47[0][0]']          
                                                                                                  
 conv2d_52 (Conv2D)             (None, 4, 4, 256)    33024       ['activation_45[0][0]']          
                                                                                                  
 batch_normalization_51 (BatchN  (None, 4, 4, 256)   1024        ['conv2d_51[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 batch_normalization_52 (BatchN  (None, 4, 4, 256)   1024        ['conv2d_52[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 add_15 (Add)                   (None, 4, 4, 256)    0           ['batch_normalization_51[0][0]', 
                                                                  'batch_normalization_52[0][0]'] 
                                                                                                  
 activation_48 (Activation)     (None, 4, 4, 256)    0           ['add_15[0][0]']                 
                                                                                                  
 conv2d_53 (Conv2D)             (None, 4, 4, 64)     16448       ['activation_48[0][0]']          
                                                                                                  
 batch_normalization_53 (BatchN  (None, 4, 4, 64)    256         ['conv2d_53[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_49 (Activation)     (None, 4, 4, 64)     0           ['batch_normalization_53[0][0]'] 
                                                                                                  
 conv2d_54 (Conv2D)             (None, 4, 4, 64)     36928       ['activation_49[0][0]']          
                                                                                                  
 batch_normalization_54 (BatchN  (None, 4, 4, 64)    256         ['conv2d_54[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_50 (Activation)     (None, 4, 4, 64)     0           ['batch_normalization_54[0][0]'] 
                                                                                                  
 conv2d_55 (Conv2D)             (None, 4, 4, 256)    16640       ['activation_50[0][0]']          
                                                                                                  
 batch_normalization_55 (BatchN  (None, 4, 4, 256)   1024        ['conv2d_55[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 add_16 (Add)                   (None, 4, 4, 256)    0           ['batch_normalization_55[0][0]', 
                                                                  'activation_48[0][0]']          
                                                                                                  
 activation_51 (Activation)     (None, 4, 4, 256)    0           ['add_16[0][0]']                 
                                                                                                  
 conv2d_56 (Conv2D)             (None, 4, 4, 64)     16448       ['activation_51[0][0]']          
                                                                                                  
 batch_normalization_56 (BatchN  (None, 4, 4, 64)    256         ['conv2d_56[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_52 (Activation)     (None, 4, 4, 64)     0           ['batch_normalization_56[0][0]'] 
                                                                                                  
 conv2d_57 (Conv2D)             (None, 4, 4, 64)     36928       ['activation_52[0][0]']          
                                                                                                  
 batch_normalization_57 (BatchN  (None, 4, 4, 64)    256         ['conv2d_57[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_53 (Activation)     (None, 4, 4, 64)     0           ['batch_normalization_57[0][0]'] 
                                                                                                  
 conv2d_58 (Conv2D)             (None, 4, 4, 256)    16640       ['activation_53[0][0]']          
                                                                                                  
 batch_normalization_58 (BatchN  (None, 4, 4, 256)   1024        ['conv2d_58[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 add_17 (Add)                   (None, 4, 4, 256)    0           ['batch_normalization_58[0][0]', 
                                                                  'activation_51[0][0]']          
                                                                                                  
 activation_54 (Activation)     (None, 4, 4, 256)    0           ['add_17[0][0]']                 
                                                                                                  
 conv2d_59 (Conv2D)             (None, 4, 4, 64)     16448       ['activation_54[0][0]']          
                                                                                                  
 batch_normalization_59 (BatchN  (None, 4, 4, 64)    256         ['conv2d_59[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_55 (Activation)     (None, 4, 4, 64)     0           ['batch_normalization_59[0][0]'] 
                                                                                                  
 conv2d_60 (Conv2D)             (None, 4, 4, 64)     36928       ['activation_55[0][0]']          
                                                                                                  
 batch_normalization_60 (BatchN  (None, 4, 4, 64)    256         ['conv2d_60[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_56 (Activation)     (None, 4, 4, 64)     0           ['batch_normalization_60[0][0]'] 
                                                                                                  
 conv2d_61 (Conv2D)             (None, 4, 4, 256)    16640       ['activation_56[0][0]']          
                                                                                                  
 batch_normalization_61 (BatchN  (None, 4, 4, 256)   1024        ['conv2d_61[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 add_18 (Add)                   (None, 4, 4, 256)    0           ['batch_normalization_61[0][0]', 
                                                                  'activation_54[0][0]']          
                                                                                                  
 activation_57 (Activation)     (None, 4, 4, 256)    0           ['add_18[0][0]']                 
                                                                                                  
 conv2d_62 (Conv2D)             (None, 4, 4, 64)     16448       ['activation_57[0][0]']          
                                                                                                  
 batch_normalization_62 (BatchN  (None, 4, 4, 64)    256         ['conv2d_62[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_58 (Activation)     (None, 4, 4, 64)     0           ['batch_normalization_62[0][0]'] 
                                                                                                  
 conv2d_63 (Conv2D)             (None, 4, 4, 64)     36928       ['activation_58[0][0]']          
                                                                                                  
 batch_normalization_63 (BatchN  (None, 4, 4, 64)    256         ['conv2d_63[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_59 (Activation)     (None, 4, 4, 64)     0           ['batch_normalization_63[0][0]'] 
                                                                                                  
 conv2d_64 (Conv2D)             (None, 4, 4, 256)    16640       ['activation_59[0][0]']          
                                                                                                  
 batch_normalization_64 (BatchN  (None, 4, 4, 256)   1024        ['conv2d_64[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 add_19 (Add)                   (None, 4, 4, 256)    0           ['batch_normalization_64[0][0]', 
                                                                  'activation_57[0][0]']          
                                                                                                  
 activation_60 (Activation)     (None, 4, 4, 256)    0           ['add_19[0][0]']                 
                                                                                                  
 average_pooling2d (AveragePool  (None, 2, 2, 256)   0           ['activation_60[0][0]']          
 ing2D)                                                                                           
                                                                                                  
 flatten (Flatten)              (None, 1024)         0           ['average_pooling2d[0][0]']      
                                                                                                  
 dense (Dense)                  (None, 256)          262400      ['flatten[0][0]']                
                                                                                                  
 dense_1 (Dense)                (None, 256)          65792       ['dense[0][0]']                  
                                                                                                  
 dropout (Dropout)              (None, 256)          0           ['dense_1[0][0]']                
                                                                                                  
 dense_2 (Dense)                (None, 15)           3855        ['dropout[0][0]']                
                                                                                                  
==================================================================================================
Total params: 852,863
Trainable params: 844,575
Non-trainable params: 8,288
__________________________________________________________________________________________________

CustomResnet_31.summary()

Model: "ResNet50"
__________________________________________________________________________________________________
 Layer (type)                   Output Shape         Param #     Connected to                     
==================================================================================================
 input_1 (InputLayer)           [(None, 31, 31, 1)]  0           []                               
                                                                                                  
 conv2d (Conv2D)                (None, 16, 16, 64)   3200        ['input_1[0][0]']                
                                                                                                  
 batch_normalization (BatchNorm  (None, 16, 16, 64)  256         ['conv2d[0][0]']                 
 alization)                                                                                       
                                                                                                  
 activation (Activation)        (None, 16, 16, 64)   0           ['batch_normalization[0][0]']    
                                                                                                  
 max_pooling2d (MaxPooling2D)   (None, 7, 7, 64)     0           ['activation[0][0]']             
                                                                                                  
 conv2d_1 (Conv2D)              (None, 7, 7, 8)      520         ['max_pooling2d[0][0]']          
                                                                                                  
 batch_normalization_1 (BatchNo  (None, 7, 7, 8)     32          ['conv2d_1[0][0]']               
 rmalization)                                                                                     
                                                                                                  
 activation_1 (Activation)      (None, 7, 7, 8)      0           ['batch_normalization_1[0][0]']  
                                                                                                  
 conv2d_2 (Conv2D)              (None, 7, 7, 8)      584         ['activation_1[0][0]']           
                                                                                                  
 batch_normalization_2 (BatchNo  (None, 7, 7, 8)     32          ['conv2d_2[0][0]']               
 rmalization)                                                                                     
                                                                                                  
 activation_2 (Activation)      (None, 7, 7, 8)      0           ['batch_normalization_2[0][0]']  
                                                                                                  
 conv2d_3 (Conv2D)              (None, 7, 7, 32)     288         ['activation_2[0][0]']           
                                                                                                  
 conv2d_4 (Conv2D)              (None, 7, 7, 32)     2080        ['max_pooling2d[0][0]']          
                                                                                                  
 batch_normalization_3 (BatchNo  (None, 7, 7, 32)    128         ['conv2d_3[0][0]']               
 rmalization)                                                                                     
                                                                                                  
 batch_normalization_4 (BatchNo  (None, 7, 7, 32)    128         ['conv2d_4[0][0]']               
 rmalization)                                                                                     
                                                                                                  
 add (Add)                      (None, 7, 7, 32)     0           ['batch_normalization_3[0][0]',  
                                                                  'batch_normalization_4[0][0]']  
                                                                                                  
 activation_3 (Activation)      (None, 7, 7, 32)     0           ['add[0][0]']                    
                                                                                                  
 conv2d_5 (Conv2D)              (None, 7, 7, 8)      264         ['activation_3[0][0]']           
                                                                                                  
 batch_normalization_5 (BatchNo  (None, 7, 7, 8)     32          ['conv2d_5[0][0]']               
 rmalization)                                                                                     
                                                                                                  
 activation_4 (Activation)      (None, 7, 7, 8)      0           ['batch_normalization_5[0][0]']  
                                                                                                  
 conv2d_6 (Conv2D)              (None, 7, 7, 8)      584         ['activation_4[0][0]']           
                                                                                                  
 batch_normalization_6 (BatchNo  (None, 7, 7, 8)     32          ['conv2d_6[0][0]']               
 rmalization)                                                                                     
                                                                                                  
 activation_5 (Activation)      (None, 7, 7, 8)      0           ['batch_normalization_6[0][0]']  
                                                                                                  
 conv2d_7 (Conv2D)              (None, 7, 7, 32)     288         ['activation_5[0][0]']           
                                                                                                  
 batch_normalization_7 (BatchNo  (None, 7, 7, 32)    128         ['conv2d_7[0][0]']               
 rmalization)                                                                                     
                                                                                                  
 add_1 (Add)                    (None, 7, 7, 32)     0           ['batch_normalization_7[0][0]',  
                                                                  'activation_3[0][0]']           
                                                                                                  
 activation_6 (Activation)      (None, 7, 7, 32)     0           ['add_1[0][0]']                  
                                                                                                  
 conv2d_8 (Conv2D)              (None, 7, 7, 8)      264         ['activation_6[0][0]']           
                                                                                                  
 batch_normalization_8 (BatchNo  (None, 7, 7, 8)     32          ['conv2d_8[0][0]']               
 rmalization)                                                                                     
                                                                                                  
 activation_7 (Activation)      (None, 7, 7, 8)      0           ['batch_normalization_8[0][0]']  
                                                                                                  
 conv2d_9 (Conv2D)              (None, 7, 7, 8)      584         ['activation_7[0][0]']           
                                                                                                  
 batch_normalization_9 (BatchNo  (None, 7, 7, 8)     32          ['conv2d_9[0][0]']               
 rmalization)                                                                                     
                                                                                                  
 activation_8 (Activation)      (None, 7, 7, 8)      0           ['batch_normalization_9[0][0]']  
                                                                                                  
 conv2d_10 (Conv2D)             (None, 7, 7, 32)     288         ['activation_8[0][0]']           
                                                                                                  
 batch_normalization_10 (BatchN  (None, 7, 7, 32)    128         ['conv2d_10[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 add_2 (Add)                    (None, 7, 7, 32)     0           ['batch_normalization_10[0][0]', 
                                                                  'activation_6[0][0]']           
                                                                                                  
 activation_9 (Activation)      (None, 7, 7, 32)     0           ['add_2[0][0]']                  
                                                                                                  
 conv2d_11 (Conv2D)             (None, 7, 7, 8)      264         ['activation_9[0][0]']           
                                                                                                  
 batch_normalization_11 (BatchN  (None, 7, 7, 8)     32          ['conv2d_11[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_10 (Activation)     (None, 7, 7, 8)      0           ['batch_normalization_11[0][0]'] 
                                                                                                  
 conv2d_12 (Conv2D)             (None, 7, 7, 8)      584         ['activation_10[0][0]']          
                                                                                                  
 batch_normalization_12 (BatchN  (None, 7, 7, 8)     32          ['conv2d_12[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_11 (Activation)     (None, 7, 7, 8)      0           ['batch_normalization_12[0][0]'] 
                                                                                                  
 conv2d_13 (Conv2D)             (None, 7, 7, 32)     288         ['activation_11[0][0]']          
                                                                                                  
 batch_normalization_13 (BatchN  (None, 7, 7, 32)    128         ['conv2d_13[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 add_3 (Add)                    (None, 7, 7, 32)     0           ['batch_normalization_13[0][0]', 
                                                                  'activation_9[0][0]']           
                                                                                                  
 activation_12 (Activation)     (None, 7, 7, 32)     0           ['add_3[0][0]']                  
                                                                                                  
 conv2d_14 (Conv2D)             (None, 4, 4, 16)     528         ['activation_12[0][0]']          
                                                                                                  
 batch_normalization_14 (BatchN  (None, 4, 4, 16)    64          ['conv2d_14[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_13 (Activation)     (None, 4, 4, 16)     0           ['batch_normalization_14[0][0]'] 
                                                                                                  
 conv2d_15 (Conv2D)             (None, 4, 4, 16)     2320        ['activation_13[0][0]']          
                                                                                                  
 batch_normalization_15 (BatchN  (None, 4, 4, 16)    64          ['conv2d_15[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_14 (Activation)     (None, 4, 4, 16)     0           ['batch_normalization_15[0][0]'] 
                                                                                                  
 conv2d_16 (Conv2D)             (None, 4, 4, 64)     1088        ['activation_14[0][0]']          
                                                                                                  
 conv2d_17 (Conv2D)             (None, 4, 4, 64)     2112        ['activation_12[0][0]']          
                                                                                                  
 batch_normalization_16 (BatchN  (None, 4, 4, 64)    256         ['conv2d_16[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 batch_normalization_17 (BatchN  (None, 4, 4, 64)    256         ['conv2d_17[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 add_4 (Add)                    (None, 4, 4, 64)     0           ['batch_normalization_16[0][0]', 
                                                                  'batch_normalization_17[0][0]'] 
                                                                                                  
 activation_15 (Activation)     (None, 4, 4, 64)     0           ['add_4[0][0]']                  
                                                                                                  
 conv2d_18 (Conv2D)             (None, 4, 4, 16)     1040        ['activation_15[0][0]']          
                                                                                                  
 batch_normalization_18 (BatchN  (None, 4, 4, 16)    64          ['conv2d_18[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_16 (Activation)     (None, 4, 4, 16)     0           ['batch_normalization_18[0][0]'] 
                                                                                                  
 conv2d_19 (Conv2D)             (None, 4, 4, 16)     2320        ['activation_16[0][0]']          
                                                                                                  
 batch_normalization_19 (BatchN  (None, 4, 4, 16)    64          ['conv2d_19[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_17 (Activation)     (None, 4, 4, 16)     0           ['batch_normalization_19[0][0]'] 
                                                                                                  
 conv2d_20 (Conv2D)             (None, 4, 4, 64)     1088        ['activation_17[0][0]']          
                                                                                                  
 batch_normalization_20 (BatchN  (None, 4, 4, 64)    256         ['conv2d_20[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 add_5 (Add)                    (None, 4, 4, 64)     0           ['batch_normalization_20[0][0]', 
                                                                  'activation_15[0][0]']          
                                                                                                  
 activation_18 (Activation)     (None, 4, 4, 64)     0           ['add_5[0][0]']                  
                                                                                                  
 conv2d_21 (Conv2D)             (None, 4, 4, 16)     1040        ['activation_18[0][0]']          
                                                                                                  
 batch_normalization_21 (BatchN  (None, 4, 4, 16)    64          ['conv2d_21[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_19 (Activation)     (None, 4, 4, 16)     0           ['batch_normalization_21[0][0]'] 
                                                                                                  
 conv2d_22 (Conv2D)             (None, 4, 4, 16)     2320        ['activation_19[0][0]']          
                                                                                                  
 batch_normalization_22 (BatchN  (None, 4, 4, 16)    64          ['conv2d_22[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_20 (Activation)     (None, 4, 4, 16)     0           ['batch_normalization_22[0][0]'] 
                                                                                                  
 conv2d_23 (Conv2D)             (None, 4, 4, 64)     1088        ['activation_20[0][0]']          
                                                                                                  
 batch_normalization_23 (BatchN  (None, 4, 4, 64)    256         ['conv2d_23[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 add_6 (Add)                    (None, 4, 4, 64)     0           ['batch_normalization_23[0][0]', 
                                                                  'activation_18[0][0]']          
                                                                                                  
 activation_21 (Activation)     (None, 4, 4, 64)     0           ['add_6[0][0]']                  
                                                                                                  
 conv2d_24 (Conv2D)             (None, 4, 4, 16)     1040        ['activation_21[0][0]']          
                                                                                                  
 batch_normalization_24 (BatchN  (None, 4, 4, 16)    64          ['conv2d_24[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_22 (Activation)     (None, 4, 4, 16)     0           ['batch_normalization_24[0][0]'] 
                                                                                                  
 conv2d_25 (Conv2D)             (None, 4, 4, 16)     2320        ['activation_22[0][0]']          
                                                                                                  
 batch_normalization_25 (BatchN  (None, 4, 4, 16)    64          ['conv2d_25[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_23 (Activation)     (None, 4, 4, 16)     0           ['batch_normalization_25[0][0]'] 
                                                                                                  
 conv2d_26 (Conv2D)             (None, 4, 4, 64)     1088        ['activation_23[0][0]']          
                                                                                                  
 batch_normalization_26 (BatchN  (None, 4, 4, 64)    256         ['conv2d_26[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 add_7 (Add)                    (None, 4, 4, 64)     0           ['batch_normalization_26[0][0]', 
                                                                  'activation_21[0][0]']          
                                                                                                  
 activation_24 (Activation)     (None, 4, 4, 64)     0           ['add_7[0][0]']                  
                                                                                                  
 conv2d_27 (Conv2D)             (None, 2, 2, 32)     2080        ['activation_24[0][0]']          
                                                                                                  
 batch_normalization_27 (BatchN  (None, 2, 2, 32)    128         ['conv2d_27[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_25 (Activation)     (None, 2, 2, 32)     0           ['batch_normalization_27[0][0]'] 
                                                                                                  
 conv2d_28 (Conv2D)             (None, 2, 2, 32)     9248        ['activation_25[0][0]']          
                                                                                                  
 batch_normalization_28 (BatchN  (None, 2, 2, 32)    128         ['conv2d_28[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_26 (Activation)     (None, 2, 2, 32)     0           ['batch_normalization_28[0][0]'] 
                                                                                                  
 conv2d_29 (Conv2D)             (None, 2, 2, 128)    4224        ['activation_26[0][0]']          
                                                                                                  
 conv2d_30 (Conv2D)             (None, 2, 2, 128)    8320        ['activation_24[0][0]']          
                                                                                                  
 batch_normalization_29 (BatchN  (None, 2, 2, 128)   512         ['conv2d_29[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 batch_normalization_30 (BatchN  (None, 2, 2, 128)   512         ['conv2d_30[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 add_8 (Add)                    (None, 2, 2, 128)    0           ['batch_normalization_29[0][0]', 
                                                                  'batch_normalization_30[0][0]'] 
                                                                                                  
 activation_27 (Activation)     (None, 2, 2, 128)    0           ['add_8[0][0]']                  
                                                                                                  
 conv2d_31 (Conv2D)             (None, 2, 2, 32)     4128        ['activation_27[0][0]']          
                                                                                                  
 batch_normalization_31 (BatchN  (None, 2, 2, 32)    128         ['conv2d_31[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_28 (Activation)     (None, 2, 2, 32)     0           ['batch_normalization_31[0][0]'] 
                                                                                                  
 conv2d_32 (Conv2D)             (None, 2, 2, 32)     9248        ['activation_28[0][0]']          
                                                                                                  
 batch_normalization_32 (BatchN  (None, 2, 2, 32)    128         ['conv2d_32[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_29 (Activation)     (None, 2, 2, 32)     0           ['batch_normalization_32[0][0]'] 
                                                                                                  
 conv2d_33 (Conv2D)             (None, 2, 2, 128)    4224        ['activation_29[0][0]']          
                                                                                                  
 batch_normalization_33 (BatchN  (None, 2, 2, 128)   512         ['conv2d_33[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 add_9 (Add)                    (None, 2, 2, 128)    0           ['batch_normalization_33[0][0]', 
                                                                  'activation_27[0][0]']          
                                                                                                  
 activation_30 (Activation)     (None, 2, 2, 128)    0           ['add_9[0][0]']                  
                                                                                                  
 conv2d_34 (Conv2D)             (None, 2, 2, 32)     4128        ['activation_30[0][0]']          
                                                                                                  
 batch_normalization_34 (BatchN  (None, 2, 2, 32)    128         ['conv2d_34[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_31 (Activation)     (None, 2, 2, 32)     0           ['batch_normalization_34[0][0]'] 
                                                                                                  
 conv2d_35 (Conv2D)             (None, 2, 2, 32)     9248        ['activation_31[0][0]']          
                                                                                                  
 batch_normalization_35 (BatchN  (None, 2, 2, 32)    128         ['conv2d_35[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_32 (Activation)     (None, 2, 2, 32)     0           ['batch_normalization_35[0][0]'] 
                                                                                                  
 conv2d_36 (Conv2D)             (None, 2, 2, 128)    4224        ['activation_32[0][0]']          
                                                                                                  
 batch_normalization_36 (BatchN  (None, 2, 2, 128)   512         ['conv2d_36[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 add_10 (Add)                   (None, 2, 2, 128)    0           ['batch_normalization_36[0][0]', 
                                                                  'activation_30[0][0]']          
                                                                                                  
 activation_33 (Activation)     (None, 2, 2, 128)    0           ['add_10[0][0]']                 
                                                                                                  
 conv2d_37 (Conv2D)             (None, 2, 2, 32)     4128        ['activation_33[0][0]']          
                                                                                                  
 batch_normalization_37 (BatchN  (None, 2, 2, 32)    128         ['conv2d_37[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_34 (Activation)     (None, 2, 2, 32)     0           ['batch_normalization_37[0][0]'] 
                                                                                                  
 conv2d_38 (Conv2D)             (None, 2, 2, 32)     9248        ['activation_34[0][0]']          
                                                                                                  
 batch_normalization_38 (BatchN  (None, 2, 2, 32)    128         ['conv2d_38[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_35 (Activation)     (None, 2, 2, 32)     0           ['batch_normalization_38[0][0]'] 
                                                                                                  
 conv2d_39 (Conv2D)             (None, 2, 2, 128)    4224        ['activation_35[0][0]']          
                                                                                                  
 batch_normalization_39 (BatchN  (None, 2, 2, 128)   512         ['conv2d_39[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 add_11 (Add)                   (None, 2, 2, 128)    0           ['batch_normalization_39[0][0]', 
                                                                  'activation_33[0][0]']          
                                                                                                  
 activation_36 (Activation)     (None, 2, 2, 128)    0           ['add_11[0][0]']                 
                                                                                                  
 conv2d_40 (Conv2D)             (None, 1, 1, 64)     8256        ['activation_36[0][0]']          
                                                                                                  
 batch_normalization_40 (BatchN  (None, 1, 1, 64)    256         ['conv2d_40[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_37 (Activation)     (None, 1, 1, 64)     0           ['batch_normalization_40[0][0]'] 
                                                                                                  
 conv2d_41 (Conv2D)             (None, 1, 1, 64)     36928       ['activation_37[0][0]']          
                                                                                                  
 batch_normalization_41 (BatchN  (None, 1, 1, 64)    256         ['conv2d_41[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_38 (Activation)     (None, 1, 1, 64)     0           ['batch_normalization_41[0][0]'] 
                                                                                                  
 conv2d_42 (Conv2D)             (None, 1, 1, 256)    16640       ['activation_38[0][0]']          
                                                                                                  
 conv2d_43 (Conv2D)             (None, 1, 1, 256)    33024       ['activation_36[0][0]']          
                                                                                                  
 batch_normalization_42 (BatchN  (None, 1, 1, 256)   1024        ['conv2d_42[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 batch_normalization_43 (BatchN  (None, 1, 1, 256)   1024        ['conv2d_43[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 add_12 (Add)                   (None, 1, 1, 256)    0           ['batch_normalization_42[0][0]', 
                                                                  'batch_normalization_43[0][0]'] 
                                                                                                  
 activation_39 (Activation)     (None, 1, 1, 256)    0           ['add_12[0][0]']                 
                                                                                                  
 conv2d_44 (Conv2D)             (None, 1, 1, 64)     16448       ['activation_39[0][0]']          
                                                                                                  
 batch_normalization_44 (BatchN  (None, 1, 1, 64)    256         ['conv2d_44[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_40 (Activation)     (None, 1, 1, 64)     0           ['batch_normalization_44[0][0]'] 
                                                                                                  
 conv2d_45 (Conv2D)             (None, 1, 1, 64)     36928       ['activation_40[0][0]']          
                                                                                                  
 batch_normalization_45 (BatchN  (None, 1, 1, 64)    256         ['conv2d_45[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_41 (Activation)     (None, 1, 1, 64)     0           ['batch_normalization_45[0][0]'] 
                                                                                                  
 conv2d_46 (Conv2D)             (None, 1, 1, 256)    16640       ['activation_41[0][0]']          
                                                                                                  
 batch_normalization_46 (BatchN  (None, 1, 1, 256)   1024        ['conv2d_46[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 add_13 (Add)                   (None, 1, 1, 256)    0           ['batch_normalization_46[0][0]', 
                                                                  'activation_39[0][0]']          
                                                                                                  
 activation_42 (Activation)     (None, 1, 1, 256)    0           ['add_13[0][0]']                 
                                                                                                  
 conv2d_47 (Conv2D)             (None, 1, 1, 64)     16448       ['activation_42[0][0]']          
                                                                                                  
 batch_normalization_47 (BatchN  (None, 1, 1, 64)    256         ['conv2d_47[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_43 (Activation)     (None, 1, 1, 64)     0           ['batch_normalization_47[0][0]'] 
                                                                                                  
 conv2d_48 (Conv2D)             (None, 1, 1, 64)     36928       ['activation_43[0][0]']          
                                                                                                  
 batch_normalization_48 (BatchN  (None, 1, 1, 64)    256         ['conv2d_48[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_44 (Activation)     (None, 1, 1, 64)     0           ['batch_normalization_48[0][0]'] 
                                                                                                  
 conv2d_49 (Conv2D)             (None, 1, 1, 256)    16640       ['activation_44[0][0]']          
                                                                                                  
 batch_normalization_49 (BatchN  (None, 1, 1, 256)   1024        ['conv2d_49[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 add_14 (Add)                   (None, 1, 1, 256)    0           ['batch_normalization_49[0][0]', 
                                                                  'activation_42[0][0]']          
                                                                                                  
 activation_45 (Activation)     (None, 1, 1, 256)    0           ['add_14[0][0]']                 
                                                                                                  
 conv2d_50 (Conv2D)             (None, 1, 1, 64)     16448       ['activation_45[0][0]']          
                                                                                                  
 batch_normalization_50 (BatchN  (None, 1, 1, 64)    256         ['conv2d_50[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_46 (Activation)     (None, 1, 1, 64)     0           ['batch_normalization_50[0][0]'] 
                                                                                                  
 conv2d_51 (Conv2D)             (None, 1, 1, 64)     36928       ['activation_46[0][0]']          
                                                                                                  
 batch_normalization_51 (BatchN  (None, 1, 1, 64)    256         ['conv2d_51[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_47 (Activation)     (None, 1, 1, 64)     0           ['batch_normalization_51[0][0]'] 
                                                                                                  
 conv2d_52 (Conv2D)             (None, 1, 1, 256)    16640       ['activation_47[0][0]']          
                                                                                                  
 batch_normalization_52 (BatchN  (None, 1, 1, 256)   1024        ['conv2d_52[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 add_15 (Add)                   (None, 1, 1, 256)    0           ['batch_normalization_52[0][0]', 
                                                                  'activation_45[0][0]']          
                                                                                                  
 activation_48 (Activation)     (None, 1, 1, 256)    0           ['add_15[0][0]']                 
                                                                                                  
 average_pooling2d (AveragePool  (None, 1, 1, 256)   0           ['activation_48[0][0]']          
 ing2D)                                                                                           
                                                                                                  
 flatten (Flatten)              (None, 256)          0           ['average_pooling2d[0][0]']      
                                                                                                  
 dense (Dense)                  (None, 256)          65792       ['flatten[0][0]']                
                                                                                                  
 dense_1 (Dense)                (None, 256)          65792       ['dense[0][0]']                  
                                                                                                  
 dropout (Dropout)              (None, 256)          0           ['dense_1[0][0]']                
                                                                                                  
 dense_2 (Dense)                (None, 15)           3855        ['dropout[0][0]']                
                                                                                                  
==================================================================================================
Total params: 560,175
Trainable params: 553,327
Non-trainable params: 6,848
__________________________________________________________________________________________________

Training without Data Augmentation

%%time
CustomResnet_128.fit(dataset_train_128, epochs=150, validation_data=dataset_test_128, class_weight=class_weights_dict,callbacks=[EarlyStopping(monitor='val_loss', patience=10, verbose=1, mode='min')],steps_per_epoch=steps_per_epoch)

Epoch 1/150
142/142 [==============================] - 13s 58ms/step - loss: 0.4761 - accuracy: 0.1004 - f1_m: 0.0115 - val_loss: 2.9514 - val_accuracy: 0.0893 - val_f1_m: 6.5466e-04
Epoch 2/150
142/142 [==============================] - 7s 51ms/step - loss: 0.4458 - accuracy: 0.1661 - f1_m: 0.0242 - val_loss: 2.9044 - val_accuracy: 0.1473 - val_f1_m: 6.5466e-04
Epoch 3/150
142/142 [==============================] - 7s 51ms/step - loss: 0.4218 - accuracy: 0.2161 - f1_m: 0.0424 - val_loss: 2.5724 - val_accuracy: 0.2683 - val_f1_m: 0.0701
Epoch 4/150
142/142 [==============================] - 8s 52ms/step - loss: 0.3962 - accuracy: 0.2862 - f1_m: 0.0856 - val_loss: 2.3151 - val_accuracy: 0.3487 - val_f1_m: 0.1433
Epoch 5/150
142/142 [==============================] - 7s 51ms/step - loss: 0.3714 - accuracy: 0.3437 - f1_m: 0.1325 - val_loss: 2.1744 - val_accuracy: 0.3973 - val_f1_m: 0.1900
Epoch 6/150
142/142 [==============================] - 7s 51ms/step - loss: 0.3479 - accuracy: 0.3979 - f1_m: 0.2236 - val_loss: 2.0051 - val_accuracy: 0.4347 - val_f1_m: 0.3113
Epoch 7/150
142/142 [==============================] - 7s 52ms/step - loss: 0.3245 - accuracy: 0.4612 - f1_m: 0.3182 - val_loss: 1.8416 - val_accuracy: 0.4833 - val_f1_m: 0.3894
Epoch 8/150
142/142 [==============================] - 7s 52ms/step - loss: 0.3015 - accuracy: 0.5174 - f1_m: 0.4222 - val_loss: 1.6841 - val_accuracy: 0.5350 - val_f1_m: 0.4525
Epoch 9/150
142/142 [==============================] - 7s 51ms/step - loss: 0.2809 - accuracy: 0.5754 - f1_m: 0.5038 - val_loss: 1.5543 - val_accuracy: 0.5777 - val_f1_m: 0.5341
Epoch 10/150
142/142 [==============================] - 7s 51ms/step - loss: 0.2630 - accuracy: 0.6163 - f1_m: 0.5628 - val_loss: 1.5529 - val_accuracy: 0.5783 - val_f1_m: 0.5282
Epoch 11/150
142/142 [==============================] - 7s 51ms/step - loss: 0.2449 - accuracy: 0.6526 - f1_m: 0.6263 - val_loss: 1.3853 - val_accuracy: 0.6320 - val_f1_m: 0.6078
Epoch 12/150
142/142 [==============================] - 8s 53ms/step - loss: 0.2267 - accuracy: 0.7020 - f1_m: 0.6825 - val_loss: 1.2618 - val_accuracy: 0.6687 - val_f1_m: 0.6452
Epoch 13/150
142/142 [==============================] - 8s 53ms/step - loss: 0.2126 - accuracy: 0.7278 - f1_m: 0.7108 - val_loss: 1.2249 - val_accuracy: 0.6680 - val_f1_m: 0.6640
Epoch 14/150
142/142 [==============================] - 8s 54ms/step - loss: 0.2018 - accuracy: 0.7501 - f1_m: 0.7275 - val_loss: 1.2489 - val_accuracy: 0.6700 - val_f1_m: 0.6700
Epoch 15/150
142/142 [==============================] - 8s 54ms/step - loss: 0.1865 - accuracy: 0.7822 - f1_m: 0.7703 - val_loss: 1.1241 - val_accuracy: 0.7087 - val_f1_m: 0.7144
Epoch 16/150
142/142 [==============================] - 8s 53ms/step - loss: 0.1747 - accuracy: 0.8120 - f1_m: 0.8009 - val_loss: 1.1326 - val_accuracy: 0.6927 - val_f1_m: 0.6961
Epoch 17/150
142/142 [==============================] - 7s 52ms/step - loss: 0.1646 - accuracy: 0.8201 - f1_m: 0.8121 - val_loss: 1.2012 - val_accuracy: 0.6850 - val_f1_m: 0.6900
Epoch 18/150
142/142 [==============================] - 8s 53ms/step - loss: 0.1575 - accuracy: 0.8239 - f1_m: 0.8169 - val_loss: 1.1195 - val_accuracy: 0.7080 - val_f1_m: 0.7147
Epoch 19/150
142/142 [==============================] - 7s 52ms/step - loss: 0.1459 - accuracy: 0.8533 - f1_m: 0.8495 - val_loss: 1.3270 - val_accuracy: 0.6653 - val_f1_m: 0.6679
Epoch 20/150
142/142 [==============================] - 7s 52ms/step - loss: 0.1413 - accuracy: 0.8442 - f1_m: 0.8348 - val_loss: 1.0551 - val_accuracy: 0.7160 - val_f1_m: 0.7267
Epoch 21/150
142/142 [==============================] - 7s 52ms/step - loss: 0.1495 - accuracy: 0.7660 - f1_m: 0.7558 - val_loss: 1.0417 - val_accuracy: 0.7190 - val_f1_m: 0.7256
Epoch 22/150
142/142 [==============================] - 7s 52ms/step - loss: 0.1256 - accuracy: 0.8721 - f1_m: 0.8680 - val_loss: 1.0565 - val_accuracy: 0.7280 - val_f1_m: 0.7299
Epoch 23/150
142/142 [==============================] - 7s 50ms/step - loss: 0.1331 - accuracy: 0.8029 - f1_m: 0.7957 - val_loss: 1.0204 - val_accuracy: 0.7383 - val_f1_m: 0.7399
Epoch 24/150
142/142 [==============================] - 7s 51ms/step - loss: 0.1128 - accuracy: 0.8898 - f1_m: 0.8827 - val_loss: 0.9573 - val_accuracy: 0.7557 - val_f1_m: 0.7632
Epoch 25/150
142/142 [==============================] - 7s 50ms/step - loss: 0.1194 - accuracy: 0.8258 - f1_m: 0.8238 - val_loss: 0.9153 - val_accuracy: 0.7657 - val_f1_m: 0.7716
Epoch 26/150
142/142 [==============================] - 7s 50ms/step - loss: 0.1009 - accuracy: 0.9080 - f1_m: 0.9017 - val_loss: 0.8852 - val_accuracy: 0.7787 - val_f1_m: 0.7831
Epoch 27/150
142/142 [==============================] - 7s 52ms/step - loss: 0.0956 - accuracy: 0.9119 - f1_m: 0.9056 - val_loss: 0.8321 - val_accuracy: 0.7873 - val_f1_m: 0.7881
Epoch 28/150
142/142 [==============================] - 7s 50ms/step - loss: 0.0900 - accuracy: 0.9224 - f1_m: 0.9190 - val_loss: 0.7956 - val_accuracy: 0.8040 - val_f1_m: 0.8049
Epoch 29/150
142/142 [==============================] - 7s 52ms/step - loss: 0.0836 - accuracy: 0.9405 - f1_m: 0.9348 - val_loss: 0.8559 - val_accuracy: 0.7893 - val_f1_m: 0.7966
Epoch 30/150
142/142 [==============================] - 8s 52ms/step - loss: 0.0860 - accuracy: 0.9019 - f1_m: 0.8962 - val_loss: 0.9076 - val_accuracy: 0.7630 - val_f1_m: 0.7651
Epoch 31/150
142/142 [==============================] - 8s 52ms/step - loss: 0.0804 - accuracy: 0.9178 - f1_m: 0.9137 - val_loss: 0.8036 - val_accuracy: 0.7913 - val_f1_m: 0.7998
Epoch 32/150
142/142 [==============================] - 7s 51ms/step - loss: 0.0723 - accuracy: 0.9473 - f1_m: 0.9467 - val_loss: 0.8017 - val_accuracy: 0.8047 - val_f1_m: 0.8101
Epoch 33/150
142/142 [==============================] - 7s 50ms/step - loss: 0.0659 - accuracy: 0.9677 - f1_m: 0.9657 - val_loss: 0.8256 - val_accuracy: 0.7960 - val_f1_m: 0.8046
Epoch 34/150
142/142 [==============================] - 7s 50ms/step - loss: 0.0630 - accuracy: 0.9646 - f1_m: 0.9658 - val_loss: 0.7838 - val_accuracy: 0.8097 - val_f1_m: 0.8161
Epoch 35/150
142/142 [==============================] - 7s 50ms/step - loss: 0.0589 - accuracy: 0.9776 - f1_m: 0.9732 - val_loss: 0.8475 - val_accuracy: 0.7907 - val_f1_m: 0.7969
Epoch 36/150
142/142 [==============================] - 8s 54ms/step - loss: 0.0608 - accuracy: 0.9459 - f1_m: 0.9432 - val_loss: 0.8413 - val_accuracy: 0.7987 - val_f1_m: 0.8053
Epoch 37/150
142/142 [==============================] - 7s 51ms/step - loss: 0.0567 - accuracy: 0.9567 - f1_m: 0.9570 - val_loss: 0.7988 - val_accuracy: 0.8050 - val_f1_m: 0.8106
Epoch 38/150
142/142 [==============================] - 7s 51ms/step - loss: 0.0506 - accuracy: 0.9823 - f1_m: 0.9798 - val_loss: 0.8144 - val_accuracy: 0.8013 - val_f1_m: 0.8087
Epoch 39/150
142/142 [==============================] - 8s 53ms/step - loss: 0.0484 - accuracy: 0.9819 - f1_m: 0.9793 - val_loss: 0.8590 - val_accuracy: 0.7913 - val_f1_m: 0.8000
Epoch 40/150
142/142 [==============================] - 8s 52ms/step - loss: 0.0605 - accuracy: 0.9027 - f1_m: 0.8968 - val_loss: 0.8143 - val_accuracy: 0.7783 - val_f1_m: 0.7854
Epoch 41/150
142/142 [==============================] - 8s 52ms/step - loss: 0.0475 - accuracy: 0.9597 - f1_m: 0.9567 - val_loss: 0.7993 - val_accuracy: 0.7973 - val_f1_m: 0.8011
Epoch 42/150
142/142 [==============================] - 7s 52ms/step - loss: 0.0480 - accuracy: 0.9469 - f1_m: 0.9454 - val_loss: 0.7332 - val_accuracy: 0.8047 - val_f1_m: 0.8139
Epoch 43/150
142/142 [==============================] - 8s 53ms/step - loss: 0.0408 - accuracy: 0.9788 - f1_m: 0.9782 - val_loss: 0.7212 - val_accuracy: 0.8140 - val_f1_m: 0.8202
Epoch 44/150
142/142 [==============================] - 8s 53ms/step - loss: 0.0381 - accuracy: 0.9878 - f1_m: 0.9835 - val_loss: 0.7313 - val_accuracy: 0.8193 - val_f1_m: 0.8233
Epoch 45/150
142/142 [==============================] - 7s 51ms/step - loss: 0.0366 - accuracy: 0.9874 - f1_m: 0.9845 - val_loss: 0.7812 - val_accuracy: 0.8103 - val_f1_m: 0.8183
Epoch 46/150
142/142 [==============================] - 7s 52ms/step - loss: 0.0385 - accuracy: 0.9651 - f1_m: 0.9576 - val_loss: 0.7768 - val_accuracy: 0.8113 - val_f1_m: 0.8159
Epoch 47/150
142/142 [==============================] - 7s 50ms/step - loss: 0.0375 - accuracy: 0.9601 - f1_m: 0.9557 - val_loss: 0.7497 - val_accuracy: 0.8057 - val_f1_m: 0.8141
Epoch 48/150
142/142 [==============================] - 7s 52ms/step - loss: 0.0338 - accuracy: 0.9768 - f1_m: 0.9734 - val_loss: 0.7209 - val_accuracy: 0.8207 - val_f1_m: 0.8261
Epoch 49/150
142/142 [==============================] - 7s 52ms/step - loss: 0.0375 - accuracy: 0.9442 - f1_m: 0.9430 - val_loss: 0.7580 - val_accuracy: 0.8040 - val_f1_m: 0.8121
Epoch 50/150
142/142 [==============================] - 7s 52ms/step - loss: 0.0303 - accuracy: 0.9811 - f1_m: 0.9783 - val_loss: 0.6991 - val_accuracy: 0.8280 - val_f1_m: 0.8323
Epoch 51/150
142/142 [==============================] - 8s 53ms/step - loss: 0.0305 - accuracy: 0.9733 - f1_m: 0.9705 - val_loss: 0.7451 - val_accuracy: 0.8120 - val_f1_m: 0.8191
Epoch 52/150
142/142 [==============================] - 7s 51ms/step - loss: 0.0279 - accuracy: 0.9833 - f1_m: 0.9822 - val_loss: 0.6883 - val_accuracy: 0.8263 - val_f1_m: 0.8312
Epoch 53/150
142/142 [==============================] - 7s 52ms/step - loss: 0.0254 - accuracy: 0.9907 - f1_m: 0.9893 - val_loss: 0.6565 - val_accuracy: 0.8330 - val_f1_m: 0.8402
Epoch 54/150
142/142 [==============================] - 8s 53ms/step - loss: 0.0245 - accuracy: 0.9899 - f1_m: 0.9867 - val_loss: 0.7146 - val_accuracy: 0.8300 - val_f1_m: 0.8331
Epoch 55/150
142/142 [==============================] - 7s 52ms/step - loss: 0.0327 - accuracy: 0.9344 - f1_m: 0.9324 - val_loss: 0.7435 - val_accuracy: 0.8150 - val_f1_m: 0.8175
Epoch 56/150
142/142 [==============================] - 7s 52ms/step - loss: 0.0243 - accuracy: 0.9804 - f1_m: 0.9769 - val_loss: 0.7419 - val_accuracy: 0.8167 - val_f1_m: 0.8208
Epoch 57/150
142/142 [==============================] - 7s 50ms/step - loss: 0.0292 - accuracy: 0.9457 - f1_m: 0.9422 - val_loss: 0.7811 - val_accuracy: 0.8107 - val_f1_m: 0.8150
Epoch 58/150
142/142 [==============================] - 7s 50ms/step - loss: 0.0248 - accuracy: 0.9693 - f1_m: 0.9655 - val_loss: 0.7947 - val_accuracy: 0.8037 - val_f1_m: 0.8084
Epoch 59/150
142/142 [==============================] - 7s 52ms/step - loss: 0.0221 - accuracy: 0.9819 - f1_m: 0.9765 - val_loss: 0.8003 - val_accuracy: 0.8033 - val_f1_m: 0.8078
Epoch 60/150
142/142 [==============================] - 7s 52ms/step - loss: 0.0292 - accuracy: 0.9330 - f1_m: 0.9302 - val_loss: 0.7062 - val_accuracy: 0.8190 - val_f1_m: 0.8238
Epoch 61/150
142/142 [==============================] - 8s 53ms/step - loss: 0.0202 - accuracy: 0.9827 - f1_m: 0.9814 - val_loss: 0.7142 - val_accuracy: 0.8247 - val_f1_m: 0.8307
Epoch 62/150
142/142 [==============================] - 8s 54ms/step - loss: 0.0215 - accuracy: 0.9719 - f1_m: 0.9659 - val_loss: 0.7111 - val_accuracy: 0.8230 - val_f1_m: 0.8276
Epoch 63/150
142/142 [==============================] - 8s 53ms/step - loss: 0.0198 - accuracy: 0.9777 - f1_m: 0.9745 - val_loss: 0.6689 - val_accuracy: 0.8333 - val_f1_m: 0.8365
Epoch 63: early stopping
CPU times: total: 2min 55s
Wall time: 7min 54s

<keras.callbacks.History at 0x1de5c926d90>

%%time
CustomResnet_31.fit(dataset_train_31, epochs=150, validation_data=dataset_test_31, class_weight=class_weights_dict,callbacks=[EarlyStopping(monitor='val_loss', patience=10, verbose=1, mode='min')],steps_per_epoch=steps_per_epoch)

Epoch 1/150
142/142 [==============================] - 13s 42ms/step - loss: 0.4083 - accuracy: 0.0836 - f1_m: 0.0032 - val_loss: 2.9654 - val_accuracy: 0.1033 - val_f1_m: 0.0000e+00
Epoch 2/150
142/142 [==============================] - 5s 35ms/step - loss: 0.3812 - accuracy: 0.1365 - f1_m: 0.0056 - val_loss: 2.9135 - val_accuracy: 0.1187 - val_f1_m: 0.0123
Epoch 3/150
142/142 [==============================] - 5s 35ms/step - loss: 0.3607 - accuracy: 0.1728 - f1_m: 0.0203 - val_loss: 2.6146 - val_accuracy: 0.2047 - val_f1_m: 0.0141
Epoch 4/150
142/142 [==============================] - 5s 34ms/step - loss: 0.3410 - accuracy: 0.2116 - f1_m: 0.0413 - val_loss: 2.4562 - val_accuracy: 0.2567 - val_f1_m: 0.0598
Epoch 5/150
142/142 [==============================] - 5s 36ms/step - loss: 0.3233 - accuracy: 0.2388 - f1_m: 0.0667 - val_loss: 2.3426 - val_accuracy: 0.2870 - val_f1_m: 0.0813
Epoch 6/150
142/142 [==============================] - 5s 37ms/step - loss: 0.3108 - accuracy: 0.2500 - f1_m: 0.0712 - val_loss: 2.2506 - val_accuracy: 0.3380 - val_f1_m: 0.1076
Epoch 7/150
142/142 [==============================] - 5s 37ms/step - loss: 0.2920 - accuracy: 0.2965 - f1_m: 0.1235 - val_loss: 2.4821 - val_accuracy: 0.2710 - val_f1_m: 0.1290
Epoch 8/150
142/142 [==============================] - 5s 36ms/step - loss: 0.2990 - accuracy: 0.2010 - f1_m: 0.0203 - val_loss: 2.5610 - val_accuracy: 0.2100 - val_f1_m: 0.0444
Epoch 9/150
142/142 [==============================] - 5s 38ms/step - loss: 0.2793 - accuracy: 0.2601 - f1_m: 0.0633 - val_loss: 2.2077 - val_accuracy: 0.3213 - val_f1_m: 0.0890
Epoch 10/150
142/142 [==============================] - 5s 36ms/step - loss: 0.2631 - accuracy: 0.3090 - f1_m: 0.1158 - val_loss: 2.0837 - val_accuracy: 0.3583 - val_f1_m: 0.1944
Epoch 11/150
142/142 [==============================] - 5s 36ms/step - loss: 0.2527 - accuracy: 0.3207 - f1_m: 0.1470 - val_loss: 2.0381 - val_accuracy: 0.3740 - val_f1_m: 0.2442
Epoch 12/150
142/142 [==============================] - 5s 38ms/step - loss: 0.2415 - accuracy: 0.3420 - f1_m: 0.1844 - val_loss: 1.9615 - val_accuracy: 0.4127 - val_f1_m: 0.2511
Epoch 13/150
142/142 [==============================] - 5s 37ms/step - loss: 0.2286 - accuracy: 0.3792 - f1_m: 0.2515 - val_loss: 1.8474 - val_accuracy: 0.4537 - val_f1_m: 0.3288
Epoch 14/150
142/142 [==============================] - 5s 36ms/step - loss: 0.2197 - accuracy: 0.3938 - f1_m: 0.2740 - val_loss: 1.8096 - val_accuracy: 0.4573 - val_f1_m: 0.3951
Epoch 15/150
142/142 [==============================] - 5s 35ms/step - loss: 0.2088 - accuracy: 0.4175 - f1_m: 0.3157 - val_loss: 1.7768 - val_accuracy: 0.4663 - val_f1_m: 0.3987
Epoch 16/150
142/142 [==============================] - 5s 34ms/step - loss: 0.1978 - accuracy: 0.4502 - f1_m: 0.3615 - val_loss: 1.7149 - val_accuracy: 0.4937 - val_f1_m: 0.4338
Epoch 17/150
142/142 [==============================] - 5s 34ms/step - loss: 0.1917 - accuracy: 0.4495 - f1_m: 0.3636 - val_loss: 1.7532 - val_accuracy: 0.4793 - val_f1_m: 0.4364
Epoch 18/150
142/142 [==============================] - 5s 34ms/step - loss: 0.1864 - accuracy: 0.4521 - f1_m: 0.3676 - val_loss: 1.6067 - val_accuracy: 0.5110 - val_f1_m: 0.4681
Epoch 19/150
142/142 [==============================] - 5s 34ms/step - loss: 0.1784 - accuracy: 0.4718 - f1_m: 0.3944 - val_loss: 1.5518 - val_accuracy: 0.5347 - val_f1_m: 0.4763
Epoch 20/150
142/142 [==============================] - 5s 35ms/step - loss: 0.1678 - accuracy: 0.5068 - f1_m: 0.4410 - val_loss: 1.5318 - val_accuracy: 0.5303 - val_f1_m: 0.4870
Epoch 21/150
142/142 [==============================] - 5s 34ms/step - loss: 0.1739 - accuracy: 0.4490 - f1_m: 0.3552 - val_loss: 1.6464 - val_accuracy: 0.5010 - val_f1_m: 0.4377
Epoch 22/150
142/142 [==============================] - 5s 34ms/step - loss: 0.1620 - accuracy: 0.4938 - f1_m: 0.4185 - val_loss: 1.5428 - val_accuracy: 0.5317 - val_f1_m: 0.4808
Epoch 23/150
142/142 [==============================] - 5s 37ms/step - loss: 0.1536 - accuracy: 0.5196 - f1_m: 0.4491 - val_loss: 1.4553 - val_accuracy: 0.5523 - val_f1_m: 0.5167
Epoch 24/150
142/142 [==============================] - 5s 36ms/step - loss: 0.1504 - accuracy: 0.5142 - f1_m: 0.4488 - val_loss: 1.4585 - val_accuracy: 0.5550 - val_f1_m: 0.5134
Epoch 25/150
142/142 [==============================] - 5s 35ms/step - loss: 0.1406 - accuracy: 0.5535 - f1_m: 0.4971 - val_loss: 1.4752 - val_accuracy: 0.5530 - val_f1_m: 0.5359
Epoch 26/150
142/142 [==============================] - 5s 35ms/step - loss: 0.1360 - accuracy: 0.5585 - f1_m: 0.5055 - val_loss: 1.4117 - val_accuracy: 0.5703 - val_f1_m: 0.5487
Epoch 27/150
142/142 [==============================] - 5s 35ms/step - loss: 0.1308 - accuracy: 0.5689 - f1_m: 0.5129 - val_loss: 1.4372 - val_accuracy: 0.5757 - val_f1_m: 0.5498
Epoch 28/150
142/142 [==============================] - 5s 35ms/step - loss: 0.1283 - accuracy: 0.5726 - f1_m: 0.5153 - val_loss: 1.3174 - val_accuracy: 0.6073 - val_f1_m: 0.5633
Epoch 29/150
142/142 [==============================] - 5s 35ms/step - loss: 0.1175 - accuracy: 0.6112 - f1_m: 0.5767 - val_loss: 1.2980 - val_accuracy: 0.5977 - val_f1_m: 0.5926
Epoch 30/150
142/142 [==============================] - 5s 35ms/step - loss: 0.1131 - accuracy: 0.6220 - f1_m: 0.5881 - val_loss: 1.2755 - val_accuracy: 0.6220 - val_f1_m: 0.6100
Epoch 31/150
142/142 [==============================] - 5s 35ms/step - loss: 0.1107 - accuracy: 0.6238 - f1_m: 0.5910 - val_loss: 1.2757 - val_accuracy: 0.6157 - val_f1_m: 0.6105
Epoch 32/150
142/142 [==============================] - 5s 35ms/step - loss: 0.1053 - accuracy: 0.6339 - f1_m: 0.6089 - val_loss: 1.2946 - val_accuracy: 0.6190 - val_f1_m: 0.6088
Epoch 33/150
142/142 [==============================] - 5s 34ms/step - loss: 0.1002 - accuracy: 0.6609 - f1_m: 0.6265 - val_loss: 1.2433 - val_accuracy: 0.6337 - val_f1_m: 0.6191
Epoch 34/150
142/142 [==============================] - 5s 35ms/step - loss: 0.1225 - accuracy: 0.5251 - f1_m: 0.4758 - val_loss: 1.7897 - val_accuracy: 0.4753 - val_f1_m: 0.4478
Epoch 35/150
142/142 [==============================] - 5s 35ms/step - loss: 0.1628 - accuracy: 0.2895 - f1_m: 0.1006 - val_loss: 2.1136 - val_accuracy: 0.3277 - val_f1_m: 0.1608
Epoch 36/150
142/142 [==============================] - 5s 35ms/step - loss: 0.1403 - accuracy: 0.3928 - f1_m: 0.2495 - val_loss: 1.8039 - val_accuracy: 0.4287 - val_f1_m: 0.3028
Epoch 37/150
142/142 [==============================] - 5s 35ms/step - loss: 0.1343 - accuracy: 0.4111 - f1_m: 0.2844 - val_loss: 1.6605 - val_accuracy: 0.4773 - val_f1_m: 0.3700
Epoch 38/150
142/142 [==============================] - 5s 36ms/step - loss: 0.1226 - accuracy: 0.4665 - f1_m: 0.3785 - val_loss: 1.5140 - val_accuracy: 0.5257 - val_f1_m: 0.4480
Epoch 39/150
142/142 [==============================] - 5s 35ms/step - loss: 0.1145 - accuracy: 0.5051 - f1_m: 0.4277 - val_loss: 1.4530 - val_accuracy: 0.5420 - val_f1_m: 0.4879
Epoch 40/150
142/142 [==============================] - 5s 34ms/step - loss: 0.1087 - accuracy: 0.5322 - f1_m: 0.4635 - val_loss: 1.4347 - val_accuracy: 0.5547 - val_f1_m: 0.5103
Epoch 41/150
142/142 [==============================] - 5s 34ms/step - loss: 0.1044 - accuracy: 0.5444 - f1_m: 0.4889 - val_loss: 1.4611 - val_accuracy: 0.5547 - val_f1_m: 0.5160
Epoch 42/150
142/142 [==============================] - 5s 36ms/step - loss: 0.1192 - accuracy: 0.4444 - f1_m: 0.3632 - val_loss: 1.5653 - val_accuracy: 0.5073 - val_f1_m: 0.4577
Epoch 43/150
142/142 [==============================] - 5s 35ms/step - loss: 0.1026 - accuracy: 0.5379 - f1_m: 0.4787 - val_loss: 1.3226 - val_accuracy: 0.5903 - val_f1_m: 0.5557
Epoch 43: early stopping
CPU times: total: 1min 18s
Wall time: 3min 47s

<keras.callbacks.History at 0x1de5c9581f0>

evaluate_and_save_model(CustomResnet_128, 'CustomResnet_128', 64,save_models=False)
evaluate_and_save_model(CustomResnet_31, 'CustomResnet_31', 64,save_models=False)

Model Name            CustomResnet_128
Epochs                              63
Batch Size                          64
Train Loss                    0.019828
Val Loss                      0.668877
Train Acc                     0.977736
Val Acc                       0.833333
[Train - Val] Acc             0.144403
[Train - Val] Loss           -0.649049
Name: CustomResnet_128, dtype: object
Model Name            CustomResnet_31
Epochs                             43
Batch Size                         64
Train Loss                   0.100193
Val Loss                     1.243305
Train Acc                    0.660944
Val Acc                      0.633667
[Train - Val] Acc            0.027277
[Train - Val] Loss          -1.143112
Name: CustomResnet_31, dtype: object

customResnet_history_128 = json.load(open('./models/CustomResnet_128_history.json', 'r'))
customResnet_history_31 = json.load(open('./models/CustomResnet_31_history.json', 'r'))
compare_curves(customResnet_history_31, customResnet_history_128, '31x31', '128x128')

Observations:

• We can see that from our loss curve, both models show signs of overfitting
• However we can see from the graph that our accuracy for 31x31, the validation curve mirrors the train accuracy curve, Why is this the case?
  • This can be because accuracy might not be a sensitive enough metric to detect overfitting in this context. and our model might be too complex for 31x31 images.

plt.plot(customResnet_history_31['f1_m'], 'o-', label='Train F1')
plt.plot(customResnet_history_31['val_f1_m'], 'o-', label='Validation F1')
plt.title('Accuracy Curve - ' + '31x31')
plt.xlabel('Epochs')
plt.ylabel('Accuracy')
plt.ylim([0, 1])
plt.legend()

<matplotlib.legend.Legend at 0x2609a994eb0>

We can see that even with plotting out the Train and validation of F1, we still see the same mirroring of train and validation.

Training with Data Augmentation

%%time
CustomResnet_128_DA = CustomResnet(input_shape=(128, 128, 1), classes=15)
CustomResnet_128_DA.compile(optimizer=SGD(learning_rate=learning_rate, momentum=momentum), loss='categorical_crossentropy', metrics=['accuracy',f1_m])
CustomResnet_128_DA.fit(train_generator_128, epochs=150, validation_data=dataset_test_128, class_weight=class_weights_dict,callbacks=[EarlyStopping(monitor='val_loss', patience=10, verbose=1, mode='min')],steps_per_epoch=steps_per_epoch)

Epoch 1/150
142/142 [==============================] - 15s 66ms/step - loss: 0.4802 - accuracy: 0.0769 - f1_m: 0.0049 - val_loss: 3.5027 - val_accuracy: 0.0763 - val_f1_m: 0.0619
Epoch 2/150
142/142 [==============================] - 8s 56ms/step - loss: 0.4518 - accuracy: 0.1279 - f1_m: 0.0069 - val_loss: 2.7846 - val_accuracy: 0.1653 - val_f1_m: 0.0300
Epoch 3/150
142/142 [==============================] - 8s 56ms/step - loss: 0.4259 - accuracy: 0.1770 - f1_m: 0.0133 - val_loss: 2.6348 - val_accuracy: 0.2270 - val_f1_m: 0.0182
Epoch 4/150
142/142 [==============================] - 8s 55ms/step - loss: 0.4022 - accuracy: 0.2296 - f1_m: 0.0305 - val_loss: 2.4979 - val_accuracy: 0.2590 - val_f1_m: 0.0570
Epoch 5/150
142/142 [==============================] - 8s 56ms/step - loss: 0.3792 - accuracy: 0.2826 - f1_m: 0.0526 - val_loss: 2.2609 - val_accuracy: 0.3463 - val_f1_m: 0.1503
Epoch 6/150
142/142 [==============================] - 8s 55ms/step - loss: 0.3551 - accuracy: 0.3439 - f1_m: 0.1247 - val_loss: 2.0796 - val_accuracy: 0.4207 - val_f1_m: 0.2273
Epoch 7/150
142/142 [==============================] - 8s 55ms/step - loss: 0.3329 - accuracy: 0.3959 - f1_m: 0.2136 - val_loss: 1.8382 - val_accuracy: 0.4853 - val_f1_m: 0.3403
Epoch 8/150
142/142 [==============================] - 8s 55ms/step - loss: 0.3126 - accuracy: 0.4476 - f1_m: 0.3010 - val_loss: 1.7053 - val_accuracy: 0.5263 - val_f1_m: 0.4253
Epoch 9/150
142/142 [==============================] - 8s 55ms/step - loss: 0.2936 - accuracy: 0.4968 - f1_m: 0.3808 - val_loss: 1.5935 - val_accuracy: 0.5543 - val_f1_m: 0.4888
Epoch 10/150
142/142 [==============================] - 8s 56ms/step - loss: 0.2770 - accuracy: 0.5308 - f1_m: 0.4510 - val_loss: 1.4483 - val_accuracy: 0.6077 - val_f1_m: 0.5558
Epoch 11/150
142/142 [==============================] - 8s 55ms/step - loss: 0.2610 - accuracy: 0.5567 - f1_m: 0.4939 - val_loss: 1.3274 - val_accuracy: 0.6397 - val_f1_m: 0.6061
Epoch 12/150
142/142 [==============================] - 8s 55ms/step - loss: 0.2446 - accuracy: 0.5967 - f1_m: 0.5467 - val_loss: 1.3001 - val_accuracy: 0.6407 - val_f1_m: 0.6115
Epoch 13/150
142/142 [==============================] - 8s 56ms/step - loss: 0.2343 - accuracy: 0.6022 - f1_m: 0.5610 - val_loss: 1.2002 - val_accuracy: 0.6797 - val_f1_m: 0.6564
Epoch 14/150
142/142 [==============================] - 8s 55ms/step - loss: 0.2179 - accuracy: 0.6470 - f1_m: 0.6135 - val_loss: 1.1435 - val_accuracy: 0.6977 - val_f1_m: 0.6774
Epoch 15/150
142/142 [==============================] - 8s 56ms/step - loss: 0.2037 - accuracy: 0.6793 - f1_m: 0.6530 - val_loss: 1.0348 - val_accuracy: 0.7233 - val_f1_m: 0.7224
Epoch 16/150
142/142 [==============================] - 8s 57ms/step - loss: 0.1949 - accuracy: 0.6854 - f1_m: 0.6666 - val_loss: 0.9449 - val_accuracy: 0.7527 - val_f1_m: 0.7468
Epoch 17/150
142/142 [==============================] - 8s 57ms/step - loss: 0.1820 - accuracy: 0.7133 - f1_m: 0.6979 - val_loss: 0.9468 - val_accuracy: 0.7507 - val_f1_m: 0.7460
Epoch 18/150
142/142 [==============================] - 8s 55ms/step - loss: 0.1721 - accuracy: 0.7349 - f1_m: 0.7240 - val_loss: 0.8697 - val_accuracy: 0.7670 - val_f1_m: 0.7705
Epoch 19/150
142/142 [==============================] - 8s 56ms/step - loss: 0.1654 - accuracy: 0.7363 - f1_m: 0.7230 - val_loss: 0.9581 - val_accuracy: 0.7383 - val_f1_m: 0.7276
Epoch 20/150
142/142 [==============================] - 8s 54ms/step - loss: 0.1555 - accuracy: 0.7558 - f1_m: 0.7428 - val_loss: 0.8126 - val_accuracy: 0.7790 - val_f1_m: 0.7827
Epoch 21/150
142/142 [==============================] - 8s 54ms/step - loss: 0.1451 - accuracy: 0.7838 - f1_m: 0.7698 - val_loss: 1.0790 - val_accuracy: 0.6997 - val_f1_m: 0.6900
Epoch 22/150
142/142 [==============================] - 8s 56ms/step - loss: 0.1405 - accuracy: 0.7775 - f1_m: 0.7677 - val_loss: 0.7700 - val_accuracy: 0.7890 - val_f1_m: 0.7933
Epoch 23/150
142/142 [==============================] - 8s 54ms/step - loss: 0.1301 - accuracy: 0.8088 - f1_m: 0.8023 - val_loss: 0.8426 - val_accuracy: 0.7680 - val_f1_m: 0.7707
Epoch 24/150
142/142 [==============================] - 8s 54ms/step - loss: 0.1248 - accuracy: 0.8089 - f1_m: 0.8008 - val_loss: 0.6923 - val_accuracy: 0.8220 - val_f1_m: 0.8179
Epoch 25/150
142/142 [==============================] - 8s 55ms/step - loss: 0.1185 - accuracy: 0.8223 - f1_m: 0.8148 - val_loss: 0.7445 - val_accuracy: 0.7937 - val_f1_m: 0.7928
Epoch 26/150
142/142 [==============================] - 8s 55ms/step - loss: 0.1121 - accuracy: 0.8302 - f1_m: 0.8228 - val_loss: 0.7817 - val_accuracy: 0.7827 - val_f1_m: 0.7868
Epoch 27/150
142/142 [==============================] - 8s 54ms/step - loss: 0.1082 - accuracy: 0.8294 - f1_m: 0.8203 - val_loss: 0.6411 - val_accuracy: 0.8253 - val_f1_m: 0.8274
Epoch 28/150
142/142 [==============================] - 8s 54ms/step - loss: 0.1036 - accuracy: 0.8336 - f1_m: 0.8310 - val_loss: 0.6928 - val_accuracy: 0.8220 - val_f1_m: 0.8231
Epoch 29/150
142/142 [==============================] - 8s 55ms/step - loss: 0.0989 - accuracy: 0.8435 - f1_m: 0.8421 - val_loss: 0.5782 - val_accuracy: 0.8440 - val_f1_m: 0.8445
Epoch 30/150
142/142 [==============================] - 8s 56ms/step - loss: 0.0920 - accuracy: 0.8612 - f1_m: 0.8596 - val_loss: 0.6425 - val_accuracy: 0.8260 - val_f1_m: 0.8286
Epoch 31/150
142/142 [==============================] - 8s 57ms/step - loss: 0.0925 - accuracy: 0.8394 - f1_m: 0.8309 - val_loss: 0.6498 - val_accuracy: 0.8237 - val_f1_m: 0.8275
Epoch 32/150
142/142 [==============================] - 8s 56ms/step - loss: 0.0886 - accuracy: 0.8435 - f1_m: 0.8368 - val_loss: 0.6247 - val_accuracy: 0.8290 - val_f1_m: 0.8357
Epoch 33/150
142/142 [==============================] - 8s 56ms/step - loss: 0.0817 - accuracy: 0.8663 - f1_m: 0.8593 - val_loss: 0.5580 - val_accuracy: 0.8497 - val_f1_m: 0.8545
Epoch 34/150
142/142 [==============================] - 8s 55ms/step - loss: 0.0810 - accuracy: 0.8573 - f1_m: 0.8509 - val_loss: 0.5990 - val_accuracy: 0.8287 - val_f1_m: 0.8321
Epoch 35/150
142/142 [==============================] - 8s 56ms/step - loss: 0.0731 - accuracy: 0.8845 - f1_m: 0.8780 - val_loss: 0.5389 - val_accuracy: 0.8593 - val_f1_m: 0.8613
Epoch 36/150
142/142 [==============================] - 8s 56ms/step - loss: 0.0706 - accuracy: 0.8850 - f1_m: 0.8834 - val_loss: 0.5002 - val_accuracy: 0.8633 - val_f1_m: 0.8678
Epoch 37/150
142/142 [==============================] - 8s 56ms/step - loss: 0.0657 - accuracy: 0.8989 - f1_m: 0.8985 - val_loss: 0.5601 - val_accuracy: 0.8403 - val_f1_m: 0.8416
Epoch 38/150
142/142 [==============================] - 8s 55ms/step - loss: 0.0619 - accuracy: 0.9081 - f1_m: 0.9040 - val_loss: 0.5197 - val_accuracy: 0.8590 - val_f1_m: 0.8591
Epoch 39/150
142/142 [==============================] - 8s 57ms/step - loss: 0.0617 - accuracy: 0.8963 - f1_m: 0.8936 - val_loss: 0.5745 - val_accuracy: 0.8407 - val_f1_m: 0.8452
Epoch 40/150
142/142 [==============================] - 8s 55ms/step - loss: 0.0584 - accuracy: 0.9041 - f1_m: 0.9029 - val_loss: 0.5357 - val_accuracy: 0.8527 - val_f1_m: 0.8558
Epoch 41/150
142/142 [==============================] - 8s 56ms/step - loss: 0.0549 - accuracy: 0.9122 - f1_m: 0.9127 - val_loss: 0.4865 - val_accuracy: 0.8647 - val_f1_m: 0.8691
Epoch 42/150
142/142 [==============================] - 8s 56ms/step - loss: 0.0522 - accuracy: 0.9175 - f1_m: 0.9162 - val_loss: 0.5436 - val_accuracy: 0.8470 - val_f1_m: 0.8512
Epoch 43/150
142/142 [==============================] - 9s 61ms/step - loss: 0.0503 - accuracy: 0.9180 - f1_m: 0.9180 - val_loss: 0.4555 - val_accuracy: 0.8767 - val_f1_m: 0.8773
Epoch 44/150
142/142 [==============================] - 8s 56ms/step - loss: 0.0491 - accuracy: 0.9145 - f1_m: 0.9131 - val_loss: 0.5480 - val_accuracy: 0.8490 - val_f1_m: 0.8505
Epoch 45/150
142/142 [==============================] - 8s 55ms/step - loss: 0.0472 - accuracy: 0.9178 - f1_m: 0.9100 - val_loss: 0.6330 - val_accuracy: 0.8157 - val_f1_m: 0.8241
Epoch 46/150
142/142 [==============================] - 8s 55ms/step - loss: 0.0478 - accuracy: 0.9054 - f1_m: 0.8988 - val_loss: 1.0532 - val_accuracy: 0.7060 - val_f1_m: 0.7177
Epoch 47/150
142/142 [==============================] - 8s 55ms/step - loss: 0.0458 - accuracy: 0.9096 - f1_m: 0.9117 - val_loss: 0.4878 - val_accuracy: 0.8560 - val_f1_m: 0.8595
Epoch 48/150
142/142 [==============================] - 8s 53ms/step - loss: 0.0413 - accuracy: 0.9272 - f1_m: 0.9205 - val_loss: 0.5603 - val_accuracy: 0.8430 - val_f1_m: 0.8471
Epoch 49/150
142/142 [==============================] - 8s 54ms/step - loss: 0.0475 - accuracy: 0.8852 - f1_m: 0.8811 - val_loss: 0.5809 - val_accuracy: 0.8353 - val_f1_m: 0.8406
Epoch 50/150
142/142 [==============================] - 8s 54ms/step - loss: 0.0397 - accuracy: 0.9176 - f1_m: 0.9187 - val_loss: 0.5045 - val_accuracy: 0.8640 - val_f1_m: 0.8673
Epoch 51/150
142/142 [==============================] - 8s 57ms/step - loss: 0.0384 - accuracy: 0.9237 - f1_m: 0.9199 - val_loss: 0.4923 - val_accuracy: 0.8670 - val_f1_m: 0.8685
Epoch 52/150
142/142 [==============================] - 8s 58ms/step - loss: 0.0357 - accuracy: 0.9297 - f1_m: 0.9279 - val_loss: 0.4998 - val_accuracy: 0.8597 - val_f1_m: 0.8638
Epoch 53/150
142/142 [==============================] - 8s 58ms/step - loss: 0.0348 - accuracy: 0.9321 - f1_m: 0.9318 - val_loss: 0.4773 - val_accuracy: 0.8703 - val_f1_m: 0.8738
Epoch 53: early stopping
CPU times: total: 2min 37s
Wall time: 7min 13s

<keras.callbacks.History at 0x260c455e790>

We will be trying to lower our 31x31 model complexity by 3 layers to see whether our model complexity does in fact cause overfitting

%%time
def CustomResnet(input_shape, classes=15):
    tf.keras.backend.clear_session()
    X_input = Input(input_shape)

    # Stage 1
    X = Conv2D(64, (7, 7), strides=(2, 2), padding='same')(X_input)
    X = BatchNormalization(axis=3)(X)
    X = Activation('relu')(X)
    X = MaxPooling2D((3, 3), strides=(2, 2))(X)

    if input_shape == (31, 31, 1):
        stages = [
            {'blocks': 3, 'filters': [16, 16, 64]},
            {'blocks': 3, 'filters': [32, 32, 128]},
            {'blocks': 3, 'filters': [64, 64, 256]}
        ]
    elif input_shape == (128, 128, 1):
        stages = [
            {'blocks': 4, 'filters': [8, 8, 32]},
            {'blocks': 4, 'filters': [16, 16, 64]},
            {'blocks': 4, 'filters': [32, 32, 128]},
            {'blocks': 4, 'filters': [64, 64, 256]}
        ]
    for i, stage in enumerate(stages):
        X = projection_block(X, f=3, filters=stage['filters'], s=1 if i == 0 else 2)
        for _ in range(stage['blocks']):
            X = identity_block(X, 3, stage['filters'])

    X = AveragePooling2D(pool_size=(2, 2), padding='same')(X)
    X = Flatten()(X)
    X = Dense(256, activation='relu',kernel_regularizer=l2(weight_decay))(X)
    X = Dense(256, activation='relu',kernel_regularizer=l2(weight_decay))(X)
    X = Dropout(0.2)(X)
    X = Dense(classes, activation='softmax')(X)

    model = Model(inputs=X_input, outputs=X, name='ResNet50')

    return model
CustomResnet_31_DA = CustomResnet(input_shape=(31, 31, 1), classes=15)
CustomResnet_31_DA.compile(optimizer=SGD(learning_rate=learning_rate, momentum=momentum), loss='categorical_crossentropy', metrics=['accuracy',f1_m])
CustomResnet_31_DA.fit(train_generator_31, epochs=150, validation_data=dataset_test_31, class_weight=class_weights_dict,callbacks=[EarlyStopping(monitor='val_loss', patience=10, verbose=1, mode='min')],steps_per_epoch=steps_per_epoch)

Epoch 1/150
142/142 [==============================] - 16s 54ms/step - loss: 0.4020 - accuracy: 0.0827 - f1_m: 0.0017 - val_loss: 2.9070 - val_accuracy: 0.0983 - val_f1_m: 0.0000e+00
Epoch 2/150
142/142 [==============================] - 4s 30ms/step - loss: 0.3803 - accuracy: 0.1347 - f1_m: 0.0032 - val_loss: 2.7777 - val_accuracy: 0.1567 - val_f1_m: 0.0045
Epoch 3/150
142/142 [==============================] - 4s 30ms/step - loss: 0.3588 - accuracy: 0.1892 - f1_m: 0.0103 - val_loss: 2.6269 - val_accuracy: 0.2173 - val_f1_m: 0.0239
Epoch 4/150
142/142 [==============================] - 4s 31ms/step - loss: 0.3376 - accuracy: 0.2466 - f1_m: 0.0202 - val_loss: 2.3906 - val_accuracy: 0.2877 - val_f1_m: 0.0542
Epoch 5/150
142/142 [==============================] - 4s 30ms/step - loss: 0.3163 - accuracy: 0.3057 - f1_m: 0.0527 - val_loss: 2.1498 - val_accuracy: 0.3673 - val_f1_m: 0.1075
Epoch 6/150
142/142 [==============================] - 4s 30ms/step - loss: 0.2948 - accuracy: 0.3641 - f1_m: 0.1308 - val_loss: 2.0523 - val_accuracy: 0.3990 - val_f1_m: 0.1920
Epoch 7/150
142/142 [==============================] - 4s 30ms/step - loss: 0.2771 - accuracy: 0.4164 - f1_m: 0.2263 - val_loss: 1.9381 - val_accuracy: 0.4210 - val_f1_m: 0.2895
Epoch 8/150
142/142 [==============================] - 4s 30ms/step - loss: 0.2592 - accuracy: 0.4654 - f1_m: 0.3156 - val_loss: 1.6936 - val_accuracy: 0.5207 - val_f1_m: 0.4316
Epoch 9/150
142/142 [==============================] - 5s 32ms/step - loss: 0.2433 - accuracy: 0.5099 - f1_m: 0.3992 - val_loss: 1.8784 - val_accuracy: 0.4533 - val_f1_m: 0.3795
Epoch 10/150
142/142 [==============================] - 5s 32ms/step - loss: 0.2295 - accuracy: 0.5387 - f1_m: 0.4520 - val_loss: 1.4615 - val_accuracy: 0.5873 - val_f1_m: 0.5293
Epoch 11/150
142/142 [==============================] - 5s 33ms/step - loss: 0.2167 - accuracy: 0.5646 - f1_m: 0.4957 - val_loss: 1.3621 - val_accuracy: 0.6283 - val_f1_m: 0.5732
Epoch 12/150
142/142 [==============================] - 5s 31ms/step - loss: 0.2029 - accuracy: 0.6014 - f1_m: 0.5516 - val_loss: 1.3961 - val_accuracy: 0.6073 - val_f1_m: 0.5802
Epoch 13/150
142/142 [==============================] - 4s 31ms/step - loss: 0.1922 - accuracy: 0.6223 - f1_m: 0.5766 - val_loss: 1.3146 - val_accuracy: 0.6310 - val_f1_m: 0.6139
Epoch 14/150
142/142 [==============================] - 4s 30ms/step - loss: 0.1826 - accuracy: 0.6399 - f1_m: 0.6051 - val_loss: 1.1879 - val_accuracy: 0.6723 - val_f1_m: 0.6505
Epoch 15/150
142/142 [==============================] - 4s 29ms/step - loss: 0.1721 - accuracy: 0.6690 - f1_m: 0.6390 - val_loss: 1.1863 - val_accuracy: 0.6673 - val_f1_m: 0.6599
Epoch 16/150
142/142 [==============================] - 4s 30ms/step - loss: 0.1669 - accuracy: 0.6685 - f1_m: 0.6409 - val_loss: 1.3799 - val_accuracy: 0.6160 - val_f1_m: 0.6137
Epoch 17/150
142/142 [==============================] - 4s 30ms/step - loss: 0.1575 - accuracy: 0.6854 - f1_m: 0.6558 - val_loss: 1.1514 - val_accuracy: 0.6747 - val_f1_m: 0.6715
Epoch 18/150
142/142 [==============================] - 4s 30ms/step - loss: 0.1487 - accuracy: 0.6995 - f1_m: 0.6791 - val_loss: 0.9902 - val_accuracy: 0.7387 - val_f1_m: 0.7298
Epoch 19/150
142/142 [==============================] - 4s 30ms/step - loss: 0.1410 - accuracy: 0.7162 - f1_m: 0.7015 - val_loss: 1.0671 - val_accuracy: 0.6973 - val_f1_m: 0.7002
Epoch 20/150
142/142 [==============================] - 4s 31ms/step - loss: 0.1342 - accuracy: 0.7323 - f1_m: 0.7138 - val_loss: 1.0853 - val_accuracy: 0.7023 - val_f1_m: 0.6925
Epoch 21/150
142/142 [==============================] - 5s 32ms/step - loss: 0.1296 - accuracy: 0.7383 - f1_m: 0.7256 - val_loss: 0.9902 - val_accuracy: 0.7260 - val_f1_m: 0.7204
Epoch 22/150
142/142 [==============================] - 5s 32ms/step - loss: 0.1222 - accuracy: 0.7533 - f1_m: 0.7387 - val_loss: 0.9093 - val_accuracy: 0.7533 - val_f1_m: 0.7534
Epoch 23/150
142/142 [==============================] - 5s 32ms/step - loss: 0.1150 - accuracy: 0.7718 - f1_m: 0.7602 - val_loss: 0.9843 - val_accuracy: 0.7267 - val_f1_m: 0.7257
Epoch 24/150
142/142 [==============================] - 5s 31ms/step - loss: 0.1162 - accuracy: 0.7381 - f1_m: 0.7232 - val_loss: 0.8945 - val_accuracy: 0.7490 - val_f1_m: 0.7532
Epoch 25/150
142/142 [==============================] - 4s 30ms/step - loss: 0.1078 - accuracy: 0.7700 - f1_m: 0.7649 - val_loss: 1.2818 - val_accuracy: 0.6430 - val_f1_m: 0.6497
Epoch 26/150
142/142 [==============================] - 4s 31ms/step - loss: 0.1008 - accuracy: 0.7940 - f1_m: 0.7836 - val_loss: 0.8308 - val_accuracy: 0.7743 - val_f1_m: 0.7727
Epoch 27/150
142/142 [==============================] - 4s 30ms/step - loss: 0.0980 - accuracy: 0.7882 - f1_m: 0.7822 - val_loss: 0.8119 - val_accuracy: 0.7757 - val_f1_m: 0.7801
Epoch 28/150
142/142 [==============================] - 4s 30ms/step - loss: 0.0909 - accuracy: 0.8121 - f1_m: 0.8072 - val_loss: 0.9431 - val_accuracy: 0.7387 - val_f1_m: 0.7402
Epoch 29/150
142/142 [==============================] - 4s 30ms/step - loss: 0.0875 - accuracy: 0.8142 - f1_m: 0.8103 - val_loss: 0.9166 - val_accuracy: 0.7347 - val_f1_m: 0.7360
Epoch 30/150
142/142 [==============================] - 4s 30ms/step - loss: 0.0925 - accuracy: 0.7709 - f1_m: 0.7588 - val_loss: 0.7437 - val_accuracy: 0.7980 - val_f1_m: 0.7959
Epoch 31/150
142/142 [==============================] - 4s 29ms/step - loss: 0.0828 - accuracy: 0.8136 - f1_m: 0.8052 - val_loss: 1.1541 - val_accuracy: 0.6770 - val_f1_m: 0.6763
Epoch 32/150
142/142 [==============================] - 4s 29ms/step - loss: 0.0851 - accuracy: 0.7789 - f1_m: 0.7720 - val_loss: 0.9351 - val_accuracy: 0.7313 - val_f1_m: 0.7298
Epoch 33/150
142/142 [==============================] - 4s 30ms/step - loss: 0.0759 - accuracy: 0.8209 - f1_m: 0.8143 - val_loss: 0.7369 - val_accuracy: 0.7993 - val_f1_m: 0.8002
Epoch 34/150
142/142 [==============================] - 4s 30ms/step - loss: 0.0720 - accuracy: 0.8347 - f1_m: 0.8344 - val_loss: 0.7423 - val_accuracy: 0.8010 - val_f1_m: 0.8054
Epoch 35/150
142/142 [==============================] - 4s 30ms/step - loss: 0.0676 - accuracy: 0.8504 - f1_m: 0.8457 - val_loss: 0.7186 - val_accuracy: 0.8117 - val_f1_m: 0.8154
Epoch 36/150
142/142 [==============================] - 4s 30ms/step - loss: 0.0642 - accuracy: 0.8579 - f1_m: 0.8552 - val_loss: 0.7594 - val_accuracy: 0.7913 - val_f1_m: 0.7921
Epoch 37/150
142/142 [==============================] - 4s 31ms/step - loss: 0.0620 - accuracy: 0.8545 - f1_m: 0.8557 - val_loss: 0.8178 - val_accuracy: 0.7627 - val_f1_m: 0.7702
Epoch 38/150
142/142 [==============================] - 5s 31ms/step - loss: 0.0664 - accuracy: 0.8254 - f1_m: 0.8183 - val_loss: 0.7707 - val_accuracy: 0.7850 - val_f1_m: 0.7825
Epoch 39/150
142/142 [==============================] - 4s 30ms/step - loss: 0.0619 - accuracy: 0.8437 - f1_m: 0.8429 - val_loss: 0.7301 - val_accuracy: 0.8013 - val_f1_m: 0.8027
Epoch 40/150
142/142 [==============================] - 4s 31ms/step - loss: 0.0567 - accuracy: 0.8622 - f1_m: 0.8589 - val_loss: 0.6947 - val_accuracy: 0.7990 - val_f1_m: 0.8061
Epoch 41/150
142/142 [==============================] - 5s 31ms/step - loss: 0.0550 - accuracy: 0.8632 - f1_m: 0.8602 - val_loss: 0.6285 - val_accuracy: 0.8273 - val_f1_m: 0.8297
Epoch 42/150
142/142 [==============================] - 5s 33ms/step - loss: 0.0528 - accuracy: 0.8680 - f1_m: 0.8677 - val_loss: 0.6695 - val_accuracy: 0.8100 - val_f1_m: 0.8194
Epoch 43/150
142/142 [==============================] - 5s 33ms/step - loss: 0.0490 - accuracy: 0.8798 - f1_m: 0.8792 - val_loss: 0.9656 - val_accuracy: 0.7410 - val_f1_m: 0.7404
Epoch 44/150
142/142 [==============================] - 5s 31ms/step - loss: 0.0490 - accuracy: 0.8802 - f1_m: 0.8746 - val_loss: 1.4523 - val_accuracy: 0.6097 - val_f1_m: 0.6154
Epoch 45/150
142/142 [==============================] - 4s 31ms/step - loss: 0.0474 - accuracy: 0.8785 - f1_m: 0.8747 - val_loss: 0.7267 - val_accuracy: 0.7960 - val_f1_m: 0.8015
Epoch 46/150
142/142 [==============================] - 5s 32ms/step - loss: 0.0440 - accuracy: 0.8926 - f1_m: 0.8858 - val_loss: 0.6655 - val_accuracy: 0.8187 - val_f1_m: 0.8213
Epoch 47/150
142/142 [==============================] - 5s 32ms/step - loss: 0.0570 - accuracy: 0.8046 - f1_m: 0.7989 - val_loss: 0.6839 - val_accuracy: 0.8057 - val_f1_m: 0.8112
Epoch 48/150
142/142 [==============================] - 4s 31ms/step - loss: 0.0468 - accuracy: 0.8624 - f1_m: 0.8567 - val_loss: 0.7186 - val_accuracy: 0.7980 - val_f1_m: 0.7965
Epoch 49/150
142/142 [==============================] - 4s 31ms/step - loss: 0.0454 - accuracy: 0.8615 - f1_m: 0.8566 - val_loss: 0.7874 - val_accuracy: 0.7750 - val_f1_m: 0.7835
Epoch 50/150
142/142 [==============================] - 4s 31ms/step - loss: 0.0416 - accuracy: 0.8818 - f1_m: 0.8790 - val_loss: 0.6543 - val_accuracy: 0.8183 - val_f1_m: 0.8244
Epoch 51/150
142/142 [==============================] - 5s 31ms/step - loss: 0.0395 - accuracy: 0.8858 - f1_m: 0.8867 - val_loss: 0.6606 - val_accuracy: 0.8170 - val_f1_m: 0.8150
Epoch 51: early stopping
CPU times: total: 1min 23s
Wall time: 4min 1s

<keras.callbacks.History at 0x2609a174190>

evaluate_and_save_model(CustomResnet_128_DA, 'CustomResnet_128_DA', 64,save_models=True)
evaluate_and_save_model(CustomResnet_31_DA, 'CustomResnet_31_DA', 64,save_models=True)

Saved trained model at ./models\CustomResnet_128_DA.h5 
Saved trained model history at ./models\CustomResnet_128_DA_history.json 
Model Name            CustomResnet_128_DA
Epochs                                 53
Batch Size                             64
Train Loss                       0.050298
Val Loss                          0.45547
Train Acc                        0.918033
Val Acc                          0.876667
[Train - Val] Acc                0.041366
[Train - Val] Loss              -0.405172
Name: CustomResnet_128_DA, dtype: object
Saved trained model at ./models\CustomResnet_31_DA.h5 
Saved trained model history at ./models\CustomResnet_31_DA_history.json 
Model Name            CustomResnet_31_DA
Epochs                                51
Batch Size                            64
Train Loss                      0.054986
Val Loss                        0.628499
Train Acc                       0.863203
Val Acc                         0.827333
[Train - Val] Acc                0.03587
[Train - Val] Loss             -0.573513
Name: CustomResnet_31_DA, dtype: object

customResnet_history_128_DA = json.load(open('./models/CustomResnet_128_DA_history.json', 'r'))
customResnet_history_31_DA = json.load(open('./models/CustomResnet_31_DA_history.json', 'r'))
compare_curves(customResnet_history_31_DA, customResnet_history_128_DA, '31x31', '128x128')

Observations:

• We can see a much better result than before data augmentation, espically for our 31x31 model. The improvement can be creditted to both reducing our model's complexity and introducing data augmentation.
• We can see improvements in both 128x128 and 31x31 in our loss curve.

Choosen model

def load_and_evaluate_model(model_name, batch_size, save_dir='./models'):
    model_history = json.load(open(os.path.join(save_dir, model_name +'_history'+ '.json'), 'r'))

    best_val_idx = np.argmax(model_history["val_accuracy"])
    result = {}
    result["Model Name"] = model_name
    result["Epochs"] = len(model_history["loss"])
    result["Batch Size"] = batch_size
    result["Train Loss"] = model_history["loss"][best_val_idx]
    result["Val Loss"] = model_history["val_loss"][best_val_idx]
    result["Train Acc"] = model_history["accuracy"][best_val_idx]
    result["Val Acc"] = model_history["val_accuracy"][best_val_idx]
    result["[Train - Val] Acc"] = result["Train Acc"] - result["Val Acc"]
    result["[Train - Val] Loss"] = result["Train Loss"] - result["Val Loss"]
    result_series = pd.Series(result, name=result["Model Name"])
    return result_series
def highlight_row(s, row_index):
    return ['background-color: green' if s.name == row_index else '' for _ in s]


evaluation_31_df = pd.DataFrame()

model_names_31 = ['BaseModel_31', 'Basemodel_31_DA', 'Basemodel_31_DA_RE', 'CustomModel_31', 'CustomModel_31_DA', 'CustomResnet_31', 'CustomResnet_31_DA']

batch_size = 64

# Load and evaluate each model and append the result to the DataFrame
for model_name in model_names_31:
    result_series = load_and_evaluate_model(model_name, batch_size)
    evaluation_31_df = pd.concat([evaluation_31_df, result_series], axis=1)

evaluation_31_df = evaluation_31_df.T.reset_index()
evaluation_31_df = evaluation_31_df.drop(['index'], axis=1)
evaluation_31_df = evaluation_31_df.style.apply(highlight_row, row_index=4, axis=1)
display(evaluation_31_df)

	Model Name	Epochs	Batch Size	Train Loss	Val Loss	Train Acc	Val Acc	[Train - Val] Acc	[Train - Val] Loss
0	BaseModel_31	77	64	0.008457	0.668227	0.949823	0.828333	0.121489	-0.659770
1	Basemodel_31_DA	94	64	0.020766	0.561303	0.882366	0.833667	0.048699	-0.540537
2	Basemodel_31_DA_RE	108	64	0.059987	0.856455	0.869406	0.761667	0.107740	-0.796468
3	CustomModel_31	107	64	0.007828	0.184748	1.000000	0.952000	0.048000	-0.176920
4	CustomModel_31_DA	70	64	0.019230	0.164061	0.994683	0.959000	0.035683	-0.144830
5	CustomResnet_31	43	64	0.100193	1.243305	0.660944	0.633667	0.027277	-1.143112
6	CustomResnet_31_DA	51	64	0.054986	0.628499	0.863203	0.827333	0.035870	-0.573513

model_names_128 = ['BaseModel_128', 'Basemodel_128_DA', 'Basemodel_128_DA_RE', 'CustomModel_128', 'CustomModel_128_DA', 'CustomResnet_128', 'CustomResnet_128_DA']

evaluation_128_df = pd.DataFrame()
for model_name in model_names_128:
    result_series = load_and_evaluate_model(model_name, batch_size)
    evaluation_128_df = pd.concat([evaluation_128_df, result_series], axis=1)

evaluation_128_df = evaluation_128_df.T.reset_index()
evaluation_128_df = evaluation_128_df.drop(['index'], axis=1)
evaluation_128_df = evaluation_128_df.style.apply(highlight_row, row_index=4, axis=1)
display(evaluation_128_df)

	Model Name	Epochs	Batch Size	Train Loss	Val Loss	Train Acc	Val Acc	[Train - Val] Acc	[Train - Val] Loss
0	BaseModel_128	33	64	0.001543	0.981181	0.995791	0.786333	0.209458	-0.979638
1	Basemodel_128_DA	55	64	0.008290	0.602679	0.957466	0.842000	0.115466	-0.594389
2	Basemodel_128_DA_RE	43	64	0.064911	0.891881	0.953146	0.777333	0.175812	-0.826971
3	CustomModel_128	68	64	0.018694	0.096466	1.000000	0.979000	0.021000	-0.077772
4	CustomModel_128_DA	85	64	0.009669	0.072805	0.995902	0.983000	0.012902	-0.063136
5	CustomResnet_128	63	64	0.019828	0.668877	0.977736	0.833333	0.144403	-0.649049
6	CustomResnet_128_DA	53	64	0.050298	0.455470	0.918033	0.876667	0.041366	-0.405172

We can see that our Custom CNN model based on VGGnet architechture trained on data augmeneted images, has the best overall results. With very good train and validation accuracy with not alot of overfitting.

Why does our custom model perform better then Resnet eventhough it is proven to be better?

This could be because our Custom Resnet model only has 12 layers (for 128x128) as compared to current Resnet models such as Resnet34 and ResNet 50. The advantage of Resnet over other models is that it can handle much deeper neural network layers due to its skip connections. However due to hardware and time limitations, we are not able to train a large Custom Resnet model of 34 to 50 layers deep. While our custom model based on VGG architechture, eventhough it has less layers, it proves to be a robust performer. While simple, this simplicity makes our model easier to train with limited hardware resources and time constraints.

Model Improvement

We will be using Keras Tuner to tune certain settings in our model. The parameters we are going to tune are:

• Weight Decay
  • allows us to find the optimal regularization strength, hoping with hyperparam tuning to strike a balance between model complexity and generalization.
• Amount of Neurons is our dense layers
  • As too few neurons may result in underfitting, while too many may lead to overfitting. So we need to find a balance of model complexity
• Learning rate
  • If the learning rate is too high, our model might overshoot the optimal weights, if it's too low, the model may converge very slowly or get stuck in local minima.

from keras_tuner import HyperModel
from tensorflow.keras import layers
from tensorflow.keras.regularizers import l2

class CustomHyperModel(HyperModel):
    def __init__(self, input_shape, num_classes):
        self.input_shape = input_shape
        self.num_classes = num_classes

    def build(self, hp):
        tf.keras.backend.clear_session()
        if self.input_shape == (31,31,1):
            num_convs = [3,3,3]
            num_filters = [64,128,256]
        elif self.input_shape == (128,128,1):
            num_convs = [2,2,2,3,3]
            num_filters = [16,32,64,128,256]
        X_input = layers.Input(self.input_shape)

        for i in range(len(num_convs)):
            if i == 0:
                X = ConvoBlock(X_input, num_convs[i], num_filters[i])
            else:
                X = ConvoBlock(X, num_convs[i], num_filters[i])

        X = layers.Flatten()(X)

        X = layers.Dense(hp.Int('dense_units', min_value=64, max_value=256, step=64), activation='relu', kernel_regularizer=l2(hp.Float("weight_decay",min_value = 0.00001,max_value=0.001)))(X)
        X = layers.Dense(hp.Int('dense_units', min_value=64, max_value=256, step=64), activation='relu', kernel_regularizer=l2(hp.Float("weight_decay",min_value = 0.00001,max_value=0.001)))(X)
        X = layers.Dropout(hp.Float('dropout_rate', min_value=0.0, max_value=0.3, step=0.1))(X)
        X = layers.Dense(self.num_classes, activation='softmax')(X)

        # Create the model
        model = tf.keras.Model(inputs=X_input, outputs=X)

        # Compile the model
        model.compile(optimizer=tf.keras.optimizers.SGD(hp.Choice('learning_rate', values=[1e-1,1e-2, 1e-3,1e-4]), momentum=0.9),
                      loss='categorical_crossentropy',
                      metrics=['accuracy'])

        return model

customTunedModel_128 = CustomHyperModel(input_shape=(128, 128, 1), num_classes=15)
customTunedModel_31 = CustomHyperModel(input_shape=(31, 31, 1), num_classes=15)

tuner_128 = kt.Hyperband(customTunedModel_128, 
                        objective='val_accuracy', 
                        max_epochs=100,
                        factor=3,
                        project_name='TunedModel_128')

tuner_31 = kt.Hyperband(customTunedModel_31,
                        objective='val_accuracy',
                        max_epochs=100,
                        factor=3,
                        project_name='TunedModel_31')

tuner_128.search(train_generator_128, epochs=100, validation_data=dataset_test_128, class_weight=class_weights_dict,steps_per_epoch=steps_per_epoch,callbacks=[EarlyStopping(monitor='val_loss', patience=10, verbose=1, mode='min')])

Trial 252 Complete [00h 10m 53s]
val_accuracy: 0.9733333587646484

Best val_accuracy So Far: 0.9853333234786987
Total elapsed time: 05h 20m 38s

tuner_128.results_summary(3)

Results summary
Results in .\TunedModel_128
Showing 3 best trials
Objective(name="val_accuracy", direction="max")

Trial 0206 summary
Hyperparameters:
dense_units: 192
weight_decay: 0.0006602248964913205
dropout_rate: 0.1
learning_rate: 0.1
tuner/epochs: 100
tuner/initial_epoch: 34
tuner/bracket: 3
tuner/round: 3
tuner/trial_id: 0204
Score: 0.9853333234786987

Trial 0145 summary
Hyperparameters:
dense_units: 256
weight_decay: 0.0003630695567053518
dropout_rate: 0.1
learning_rate: 0.1
tuner/epochs: 100
tuner/initial_epoch: 34
tuner/bracket: 4
tuner/round: 4
tuner/trial_id: 0142
Score: 0.9783333539962769

Trial 0232 summary
Hyperparameters:
dense_units: 64
weight_decay: 0.0004988939209611291
dropout_rate: 0.1
learning_rate: 0.01
tuner/epochs: 100
tuner/initial_epoch: 34
tuner/bracket: 2
tuner/round: 2
tuner/trial_id: 0227
Score: 0.9756666421890259

tuner_31.search(train_generator_31, epochs=100, validation_data=dataset_test_31, class_weight=class_weights_dict,steps_per_epoch=steps_per_epoch)

Trial 254 Complete [00h 05m 51s]
val_accuracy: 0.5139999985694885

Best val_accuracy So Far: 0.9789999723434448
Total elapsed time: 02h 42m 20s

tuner_31.results_summary(3)

Results summary
Results in .\TunedModel_31
Showing 3 best trials
Objective(name="val_accuracy", direction="max")

Trial 0234 summary
Hyperparameters:
dense_units: 256
weight_decay: 0.0008862468274995598
dropout_rate: 0.1
learning_rate: 0.1
tuner/epochs: 100
tuner/initial_epoch: 34
tuner/bracket: 2
tuner/round: 2
tuner/trial_id: 0231
Score: 0.9789999723434448

Trial 0146 summary
Hyperparameters:
dense_units: 192
weight_decay: 0.000682698997366627
dropout_rate: 0.1
learning_rate: 0.1
tuner/epochs: 100
tuner/initial_epoch: 34
tuner/bracket: 4
tuner/round: 4
tuner/trial_id: 0143
Score: 0.9779999852180481

Trial 0249 summary
Hyperparameters:
dense_units: 64
weight_decay: 0.0002798940181868567
dropout_rate: 0.2
learning_rate: 0.1
tuner/epochs: 100
tuner/initial_epoch: 0
tuner/bracket: 0
tuner/round: 0
Score: 0.9746666550636292

Observations:

• We can see that with the optimal weight decay values we optained in hyperparam tuning, our model's test accuracy has improved over 2%
• We can see that our model converges better with a bigger learning rate values, as our top 3 results for both 128x128 and 31x31 have choosen 0.1 learning rate for their parameters.

final_model_128 = tuner_128.get_best_models(num_models=1)[0]
final_model_31 = tuner_31.get_best_models(num_models=1)[0]
final_model_128.save('./models/final_model_128.h5')
final_model_31.save('./models/final_model_31.h5')

final_model_128.summary()

Model: "model"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 input_1 (InputLayer)        [(None, 128, 128, 1)]     0         
                                                                 
 conv2d (Conv2D)             (None, 128, 128, 16)      160       
                                                                 
 batch_normalization (BatchN  (None, 128, 128, 16)     64        
 ormalization)                                                   
                                                                 
 activation (Activation)     (None, 128, 128, 16)      0         
                                                                 
 conv2d_1 (Conv2D)           (None, 128, 128, 16)      2320      
                                                                 
 batch_normalization_1 (Batc  (None, 128, 128, 16)     64        
 hNormalization)                                                 
                                                                 
 activation_1 (Activation)   (None, 128, 128, 16)      0         
                                                                 
 max_pooling2d (MaxPooling2D  (None, 64, 64, 16)       0         
 )                                                               
                                                                 
 conv2d_2 (Conv2D)           (None, 64, 64, 32)        4640      
                                                                 
 batch_normalization_2 (Batc  (None, 64, 64, 32)       128       
 hNormalization)                                                 
                                                                 
 activation_2 (Activation)   (None, 64, 64, 32)        0         
                                                                 
 conv2d_3 (Conv2D)           (None, 64, 64, 32)        9248      
                                                                 
 batch_normalization_3 (Batc  (None, 64, 64, 32)       128       
 hNormalization)                                                 
                                                                 
 activation_3 (Activation)   (None, 64, 64, 32)        0         
                                                                 
 max_pooling2d_1 (MaxPooling  (None, 32, 32, 32)       0         
 2D)                                                             
                                                                 
 conv2d_4 (Conv2D)           (None, 32, 32, 64)        18496     
                                                                 
 batch_normalization_4 (Batc  (None, 32, 32, 64)       256       
 hNormalization)                                                 
                                                                 
 activation_4 (Activation)   (None, 32, 32, 64)        0         
                                                                 
 conv2d_5 (Conv2D)           (None, 32, 32, 64)        36928     
                                                                 
 batch_normalization_5 (Batc  (None, 32, 32, 64)       256       
 hNormalization)                                                 
                                                                 
 activation_5 (Activation)   (None, 32, 32, 64)        0         
                                                                 
 max_pooling2d_2 (MaxPooling  (None, 16, 16, 64)       0         
 2D)                                                             
                                                                 
 conv2d_6 (Conv2D)           (None, 16, 16, 128)       73856     
                                                                 
 batch_normalization_6 (Batc  (None, 16, 16, 128)      512       
 hNormalization)                                                 
                                                                 
 activation_6 (Activation)   (None, 16, 16, 128)       0         
                                                                 
 conv2d_7 (Conv2D)           (None, 16, 16, 128)       147584    
                                                                 
 batch_normalization_7 (Batc  (None, 16, 16, 128)      512       
 hNormalization)                                                 
                                                                 
 activation_7 (Activation)   (None, 16, 16, 128)       0         
                                                                 
 conv2d_8 (Conv2D)           (None, 16, 16, 128)       147584    
                                                                 
 batch_normalization_8 (Batc  (None, 16, 16, 128)      512       
 hNormalization)                                                 
                                                                 
 activation_8 (Activation)   (None, 16, 16, 128)       0         
                                                                 
 max_pooling2d_3 (MaxPooling  (None, 8, 8, 128)        0         
 2D)                                                             
                                                                 
 conv2d_9 (Conv2D)           (None, 8, 8, 256)         295168    
                                                                 
 batch_normalization_9 (Batc  (None, 8, 8, 256)        1024      
 hNormalization)                                                 
                                                                 
 activation_9 (Activation)   (None, 8, 8, 256)         0         
                                                                 
 conv2d_10 (Conv2D)          (None, 8, 8, 256)         590080    
                                                                 
 batch_normalization_10 (Bat  (None, 8, 8, 256)        1024      
 chNormalization)                                                
                                                                 
 activation_10 (Activation)  (None, 8, 8, 256)         0         
                                                                 
 conv2d_11 (Conv2D)          (None, 8, 8, 256)         590080    
                                                                 
 batch_normalization_11 (Bat  (None, 8, 8, 256)        1024      
 chNormalization)                                                
                                                                 
 activation_11 (Activation)  (None, 8, 8, 256)         0         
                                                                 
 max_pooling2d_4 (MaxPooling  (None, 4, 4, 256)        0         
 2D)                                                             
                                                                 
 flatten (Flatten)           (None, 4096)              0         
                                                                 
 dense (Dense)               (None, 192)               786624    
                                                                 
 dense_1 (Dense)             (None, 192)               37056     
                                                                 
 dropout (Dropout)           (None, 192)               0         
                                                                 
 dense_2 (Dense)             (None, 15)                2895      
                                                                 
=================================================================
Total params: 2,748,223
Trainable params: 2,745,471
Non-trainable params: 2,752
_________________________________________________________________

final_model_31.summary()

Model: "model"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 input_1 (InputLayer)        [(None, 31, 31, 1)]       0         
                                                                 
 conv2d (Conv2D)             (None, 31, 31, 64)        640       
                                                                 
 batch_normalization (BatchN  (None, 31, 31, 64)       256       
 ormalization)                                                   
                                                                 
 activation (Activation)     (None, 31, 31, 64)        0         
                                                                 
 conv2d_1 (Conv2D)           (None, 31, 31, 64)        36928     
                                                                 
 batch_normalization_1 (Batc  (None, 31, 31, 64)       256       
 hNormalization)                                                 
                                                                 
 activation_1 (Activation)   (None, 31, 31, 64)        0         
                                                                 
 conv2d_2 (Conv2D)           (None, 31, 31, 64)        36928     
                                                                 
 batch_normalization_2 (Batc  (None, 31, 31, 64)       256       
 hNormalization)                                                 
                                                                 
 activation_2 (Activation)   (None, 31, 31, 64)        0         
                                                                 
 max_pooling2d (MaxPooling2D  (None, 15, 15, 64)       0         
 )                                                               
                                                                 
 conv2d_3 (Conv2D)           (None, 15, 15, 128)       73856     
                                                                 
 batch_normalization_3 (Batc  (None, 15, 15, 128)      512       
 hNormalization)                                                 
                                                                 
 activation_3 (Activation)   (None, 15, 15, 128)       0         
                                                                 
 conv2d_4 (Conv2D)           (None, 15, 15, 128)       147584    
                                                                 
 batch_normalization_4 (Batc  (None, 15, 15, 128)      512       
 hNormalization)                                                 
                                                                 
 activation_4 (Activation)   (None, 15, 15, 128)       0         
                                                                 
 conv2d_5 (Conv2D)           (None, 15, 15, 128)       147584    
                                                                 
 batch_normalization_5 (Batc  (None, 15, 15, 128)      512       
 hNormalization)                                                 
                                                                 
 activation_5 (Activation)   (None, 15, 15, 128)       0         
                                                                 
 max_pooling2d_1 (MaxPooling  (None, 7, 7, 128)        0         
 2D)                                                             
                                                                 
 conv2d_6 (Conv2D)           (None, 7, 7, 256)         295168    
                                                                 
 batch_normalization_6 (Batc  (None, 7, 7, 256)        1024      
 hNormalization)                                                 
                                                                 
 activation_6 (Activation)   (None, 7, 7, 256)         0         
                                                                 
 conv2d_7 (Conv2D)           (None, 7, 7, 256)         590080    
                                                                 
 batch_normalization_7 (Batc  (None, 7, 7, 256)        1024      
 hNormalization)                                                 
                                                                 
 activation_7 (Activation)   (None, 7, 7, 256)         0         
                                                                 
 conv2d_8 (Conv2D)           (None, 7, 7, 256)         590080    
                                                                 
 batch_normalization_8 (Batc  (None, 7, 7, 256)        1024      
 hNormalization)                                                 
                                                                 
 activation_8 (Activation)   (None, 7, 7, 256)         0         
                                                                 
 max_pooling2d_2 (MaxPooling  (None, 3, 3, 256)        0         
 2D)                                                             
                                                                 
 flatten (Flatten)           (None, 2304)              0         
                                                                 
 dense (Dense)               (None, 256)               590080    
                                                                 
 dense_1 (Dense)             (None, 256)               65792     
                                                                 
 dropout (Dropout)           (None, 256)               0         
                                                                 
 dense_2 (Dense)             (None, 15)                3855      
                                                                 
=================================================================
Total params: 2,583,951
Trainable params: 2,581,263
Non-trainable params: 2,688
_________________________________________________________________

Final Model

Now, we are checking how well our final model works. To make sure it's good at handling new data, we want the accuracy on the validation dataset to be similar to what we saw on the test set. This helps us trust that the model is not just good on the data it learned from but also on new, unseen examples. :)

Evaluating our Model on our validation dataset

final_model_128 = tf.keras.models.load_model('./models/final_model_128.h5')
final_model_31 = tf.keras.models.load_model('./models/final_model_31.h5')

WARNING:tensorflow:Error in loading the saved optimizer state. As a result, your model is starting with a freshly initialized optimizer.
WARNING:tensorflow:Error in loading the saved optimizer state. As a result, your model is starting with a freshly initialized optimizer.

final_model_128.evaluate(dataset_validation_128)

47/47 [==============================] - 1s 19ms/step - loss: 0.0659 - accuracy: 0.9837

[0.06585882604122162, 0.9836666584014893]

final_model_31.evaluate(dataset_validation_31)

47/47 [==============================] - 1s 12ms/step - loss: 0.0692 - accuracy: 0.9820

[0.06922943890094757, 0.9819999933242798]

We can see that our accuracy on the validation dataset is 98.37% for our 128x128 model and 98.20% for our 31x31 model. These results are very close to our test dataset results, especially for our 128x128 model. With hyperparemeter tuning, our 31x31 model improves over 3% on the validation data. This shows that our model not only does not overfit it is able to generalise very well to unseen data. :)

Classification Report

# Not shuffling dataset to avoid confusion in the classification report
dataset_validation_128 = dataset(directory_validation, (128, 128), shuffle=False)
dataset_validation_31 = dataset(directory_validation, (31, 31), shuffle=False)

# Assuming `class_labels` is a list of your class labels
class_labels = [str(label) for label in class_labels]

y_true_128 = np.concatenate([y for x, y in dataset_validation_128], axis=0)
y_true_31 = np.concatenate([y for x, y in dataset_validation_31], axis=0)
test_preds_probs_128 = final_model_128.predict(dataset_validation_128)
test_preds_probs_31 = final_model_31.predict(dataset_validation_31)
test_pred_128 = np.argmax(test_preds_probs_128, axis=1)
test_pred_31 = np.argmax(test_preds_probs_31, axis=1)

report_128 = classification_report(
    np.argmax(y_true_128, axis=1), test_pred_128, target_names=class_labels
)

report_31 = classification_report(
    np.argmax(y_true_31, axis=1), test_pred_31, target_names=class_labels
)
print("Classification Report (128x128):")
print(report_128)

Found 3000 files belonging to 15 classes.
Found 3000 files belonging to 15 classes.
47/47 [==============================] - 1s 22ms/step
47/47 [==============================] - 0s 8ms/step
Classification Report (128x128):
              precision    recall  f1-score   support

        Bean       0.99      0.98      0.99       200
Bitter Gourd       0.99      0.99      0.99       200
Bottle Gourd       0.99      0.98      0.98       200
     Brinjal       0.99      0.99      0.99       200
    Broccoli       1.00      0.98      0.99       200
     Cabbage       0.99      0.98      0.98       200
    Capsicum       0.98      0.99      0.99       200
      Carrot       0.97      1.00      0.99       200
 Cauliflower       0.99      1.00      0.99       200
    Cucumber       0.98      0.99      0.98       200
      Papaya       0.99      0.94      0.97       200
      Potato       0.95      0.99      0.97       200
     Pumpkin       0.96      0.98      0.97       200
      Radish       0.99      0.98      0.99       200
      Tomato       0.99      0.96      0.98       200

    accuracy                           0.98      3000
   macro avg       0.98      0.98      0.98      3000
weighted avg       0.98      0.98      0.98      3000

print("Classification Report (31x31):")
print(report_31)

Classification Report (31x31):
              precision    recall  f1-score   support

        Bean       0.98      0.98      0.98       200
Bitter Gourd       0.99      0.99      0.99       200
Bottle Gourd       1.00      0.98      0.99       200
     Brinjal       0.98      0.97      0.97       200
    Broccoli       0.97      0.98      0.98       200
     Cabbage       0.97      0.96      0.97       200
    Capsicum       0.99      0.99      0.99       200
      Carrot       0.99      0.99      0.99       200
 Cauliflower       0.98      0.98      0.98       200
    Cucumber       0.99      0.99      0.99       200
      Papaya       0.97      0.98      0.98       200
      Potato       0.99      0.97      0.98       200
     Pumpkin       0.98      0.99      0.98       200
      Radish       0.99      0.96      0.97       200
      Tomato       0.98      0.98      0.98       200

    accuracy                           0.98      3000
   macro avg       0.98      0.98      0.98      3000
weighted avg       0.98      0.98      0.98      3000

From looking at the classification report, we can see that

• The distribution of classes in our validation dataset is perfectly balanced
• Our model does generalise to our classes very well and do not appear to be biased to any 1 class.
• The overall accuracy of 98% in both models indicates that they perform exceptionally well in correctly classifying vegetable images. :)

Error Analysis

We can look and analyze our model's mistakes

import random
incorrect_indices_128 = np.nonzero(test_pred_128 != np.argmax(y_true_128, axis=1))[0]
incorrect_indices_31 = np.nonzero(test_pred_31 != np.argmax(y_true_31, axis=1))[0]

incorrect_images_128 = np.concatenate([x for x, y in dataset_validation_128], axis=0)[incorrect_indices_128]
incorrect_images_31 = np.concatenate([x for x, y in dataset_validation_31], axis=0)[incorrect_indices_31]

incorrect_data_128 = list(zip(incorrect_indices_128, incorrect_images_128))
random.shuffle(incorrect_data_128)
incorrect_indices_128, incorrect_images_128 = zip(*incorrect_data_128)

incorrect_data_31 = list(zip(incorrect_indices_31, incorrect_images_31))
random.shuffle(incorrect_data_31)
incorrect_indices_31, incorrect_images_31 = zip(*incorrect_data_31)

# Now you can plot the shuffled incorrect images as before
fig, axes = plt.subplots(nrows=3, ncols=5, figsize=(15, 9))
for i, ax in enumerate(axes.flat):
    if i < len(incorrect_indices_128):
        ax.imshow(incorrect_images_128[i], cmap='gray')
        ax.set_title(f"Pred: {class_labels[test_pred_128[incorrect_indices_128[i]]]}, True: {class_labels[np.argmax(y_true_128[incorrect_indices_128[i]])]}")
    ax.axis('off')
plt.tight_layout()
plt.show()

When looking at the sample examples of our 128x128 model's predictions:

• It appears that the model's mistakes are sometimes quite dumb, where it mistakes a papaya as a carrot.
• This suggests that we need to make our 128x128 model deeper and more complex to extract more prominent features from our vegatables.
• However some of its mistakes are reasonable, such as mistaking papayas for potato in black and white. Which highlights the impact of factors beyond our model's architecture, such as variations in color and image quality.

fig, axes = plt.subplots(nrows=3, ncols=5, figsize=(15, 9))
for i, ax in enumerate(axes.flat):
    if i < len(incorrect_indices_31):
        ax.imshow(incorrect_images_31[i], cmap='gray')
        ax.set_title(f"Pred: {class_labels[test_pred_31[incorrect_indices_31[i]]]}, True: {class_labels[np.argmax(y_true_31[incorrect_indices_31[i]])]}")
    ax.axis('off')
plt.tight_layout()
plt.show()

When looking at the sample examples of our 31x31 model's predictions:

• The low pixel resolution makes it ambguious even to a human what the correct class is.
• At such a size the details and distinct features that characterize different vegetables might be reduced or even lost.

So it is reasonable for the model to make such mistakes. Thus our model is doing quite well as some errors are because the images are naturally hard to interpret.

Final Model Visualization

visualkeras.layered_view(final_model_128,legend=True, to_file="./images/final_model_128.png")

visualkeras.layered_view(final_model_31,legend=True, to_file="./images/final_model_31.png")

Summary

To close off this project, this CNN project has been a great learning experience. The main focus was tweaking models for different picture sizes, like 31x31 and 128x128 pixels. Dealing with TensorFlow's batch dataset brought its own set of challenges, and getting everything just right took some time. Experimenting with different model architechture. This project taught me alot about CNNs and its architechture,while it wasn't always smooth sailing, these challenges gave me practical insights into how deep learning works in the real world.

References

Adam - Cornell University Computational Optimization Open Textbook - Optimization Wiki (no date). https://optimization.cbe.cornell.edu/index.php?title=Adam.

Zhou, P. (2020) Towards theoretically understanding why SGD generalizes better than ADAM in deep learning. https://arxiv.org/abs/2010.05627.

Simonyan, K. (2014) Very deep convolutional networks for Large-Scale image recognition. https://arxiv.org/abs/1409.1556.

He, K. (2015) Deep residual learning for image recognition. https://arxiv.org/abs/1512.03385.